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ABSTRACT 


This study was designed to examine the perceptions of Alberta 
nursing instructors regarding actual and preferred evaluators, data 
gathering practices and criteria for assessing their teaching 
effectiveness. 

The Nursing Instructor Evaluation Instrument was distributed to 
Alberta nursing instructors who were teaching one-half time or more 
in diploma or basic baccalaureate programs at the time of study. The 
first section of the questionnaire requested personal and professional 
information on nine variables. The second section consisted of seven 
types of evaluators, twelve data gathering practices and thirty evalua- 
tion criteria and requested scoring of these items in terms of their 
actual and preferred importance in evaluating teaching effectiveness. 
The criteria included had been modified from those utilized in a 
pilot study. The third section invited instructors to make comments 
concerning evaluation and/or the study. 

The statistical procedures used to analyze the data included 
correlations to determine the relationships between the nine indepen- 
dent variables; standard deviations and rankings of means to show 
the extent of common perceptions in the actual and preferred 
situations; Spearman rho calculations to determine the extent of 
similarities or differences between the ratings for the actual and 
preferred situations; T tests to determine whether actual and 
preferred differences were of statistical significance; T and F 


tests to determine the effect which the independent variables had 
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upon perceptions concerning actual and preferred evaluators and 

data gathering practices; a factor analysis to determine if criteria 
could be classified according to product, process and presage 
categorizations and T and F tests to determine the effect of the 
independent variables upon preferences for product, process or 
presage criteria. 

Analysis of the data revealed that senior administrators and 
immediate supervisors were seen as the most important evaluators while 
the teachers preferred that instructors themselves and immediate 
Supervisors be most important. Nursing instructors prepared at the 
master's level and those teaching in baccalaureate programs had the 
strongest preference for the use of peer evaluation. The instructors 
did rot see or prefer students to be a major source of evaluative 
input, however, those instructors working in baccalaureate programs 
and those prepared at the master's level saw and preferred more 
student involvement than did other Alberta nursing instructors. The 
instructors preferred that a broader range of data gathering practices 
and criteria be utilized in evaluating their teaching effectiveness. 
The preference was for data gathering practices which involved the 
direct observation of the teacher. Criteria involving evaluative and 
communicative skills were considered important. The use of product 
criteria received minimal emphasis in the actual and preferred situa- 
tions although the preference for the use of product criteria increased 
directly with the amount of teaching experience. 

The study seemed to show a need for further assessment of the 
roles which various personnel] might play in Alberta nursing 


instructor evaluation, development and use of practices to directly 
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observe the nursing instructor and the possibility of constructing 
evaluation instruments based upon criteria which were identified 


as important. 
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CHAPTER I 
STATEMENT OF THE PROBLEM AND ITS SIGNIFICANCE 
INTRODUCTION 


The concept of accountability is being emphasized increasingly 
within the nursing profession today. Part of that emphasis is reflected 
by a growing concern for the evaluation of those who practice nursing 
(Hauser, 1975). In addition, attention is being devoted to improving 
the procedures for evaluating nursing students. It would seem, however, 
that relatively little attention has been given to the topic of nursing 
instructor evaluation to date. 

Evaluation of teaching effectiveness can be viewed as a 
complex, continuously occurring process. The assessment of nursing 
instructors may be especially complex since many nursing teachers 
function in both classroom and nursing practice settings. Selection 
of appropriate evaluative criteria and practices becomes a challenging 
task. Bolton (1973:22-23) commented on the continuous manner in 
which teachers are evaluated, indicating: 

All teachers are evaluated. Regardless of how formal the 

evaluation system is, what evidence is collected and analyzed, 
or how often formal reports are written, teachers are evaluated 
often. They are evaluated by students, parents, other teachers, 
administrators, supervisors, and the public. The question is 
not whether teachers should be evaluated, since this cannot 

be avoided, but rather how systematic the evaluation should be 
in order to be most effective. 


Much of the research conducted on nursing instructor 


evaluation seems to focus on establishing criteria which distinguish 
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effective from ineffective teaching behaviors. Nursing students 

seem to be the most frequent source of input in this research. It 
seems appropriate to ascertain in more detail what nursing instructors 
themselves perceive concerning various aspects of the process involved 
in evaluating their teaching effectiveness. 

The present situation in Alberta is that nursing educators 
function in a variety of college, hospital and university settings. 
Although no move has been made to date, the Department of Advanced 
Education announced its intention in 1973 to have all schools of 
nursing come under its jurisdiction. In addition, the Department of 
Advanced Education is studying a recommendation for the phasing 
in of a plan which would mean that all nurses prepared in the 
Province of Alberta after 1990 would require a Baccalaureate degree 
in nursing. Implementation of this latter measure could have 
implications for nurse educators in terms of cooperation in program 
development and a variety of other issues. Involving Alberta nursing 
educators in a study concerning evaluation of their teaching effec- 


tiveness would seem to be a timely endeavor. 
THE PROBLEM 


The purpose of the study was to examine the perceptions of 
selected Alberta nursing educators concerning actual and preferred 
practices and criteria for evaluating their teaching effectiveness. 

Emerging from the basic problem of the study were several 
subprob lems: 

1. To what degree do nursing instructors have common 


perceptions concerning who evaluates and who should evaluate their 
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teaching effectiveness? 

2. To what degree do statistically significant differences 
exist between the perceptions concerning actual and preferred evaluators? 

3. How do the personal and professional variables of age, 
educational preparation, teaching experience, length of present 
employment, type of present employment, hours of present employment, 
area(s) of major teaching responsibility, type of program, and 
number of full-time faculty affect nursing instructor perceptions 
concerning actual and preferred evaluators? 

4. To what degree do nursing instructors have common 
perceptions concerning the data gathering practices utilized and 
preferred for the evaluation of their teaching effectiveness? 

5. To what degree do statistically significant differences 
exist between the perceptions concerning actual and preferred data 
gathering practices? 

6. How do the selected personal and professional variables 
affect nursing instructor perceptions concerning actual and preferred 
data gathering practices? 

7. To what degree do nursing instructors have common 
perceptions concerning the criteria utilized and preferred for the 
evaluation of their teaching effectiveness? 

8. To what degree do statistically significant differences 
exist between the perceptions concerning actual and preferred evaluation 
criteria? 

9. To what degree do nursing instructors have common 
perceptions concerning the present and preferred emphasis on product, 


process or presage criteria for evaluating their teaching effectiveness? 
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10. How do the selected personal and professional variables 
affect nursing instructor perceptions concerning their preferences for 


the use of product, process and presage criteria? 
IMPORTANCE OF THE STUDY 


The study should have value in that it will update and add 
to the rather limited amount of research which has been done on nursing 
instructor evaluation in Canada. The information should be of interest 
to those concerned with evaluation of nursing teachers as an educational 
issue. In addition, the study is believed to be the first of its 
kind to be conducted with Alberta nursing educators; it should have 
merit in that it will provide some baseline data concerning nursing 
instructor evaluation in this province. In particular, the study will 
provide an indication of the degree to which Alberta nursing educators 
see evaluation of their teaching effectiveness as a major concern. In 
addition, it will provide information about the extent. to which common 
practices and criteria exist and are preferred by the nursing instructors 
participating in the research. 

The study should stimulate interest in the topic of nursing 
instructor evaluation. The study will identify and examine the 
perceptions of participants concerning particular aspects of nursing 
instructor evaluation. In addition, the study will develop comparisons 
of the perceptions of nurse educators who were involved in the research. 
The analysis of the relationship between instructors’ perceptions 
regarding evaluation and the personal and professional variables 
included in the study may be of interest to those administrators and 


instructors who are involved in the development or modification of an 
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evaluation program. 

The study should be of value in that it will allow certain 
comparisons to be made between practices and criteria utilized for 
evaluation in general education and those utilized and preferred in 
the evaluation of Alberta nursing instructors. Finally, since the 
study is intended to be general in nature and cover a ranae of 
practices and criteria, the findings may stimulate research of a more 


specific nature. 
DEFINITION OF TERMS 


Teacher evaluation is an assessment of teacher effectiveness 
by measurement, rating, or ranking. It involves value judgments 
based on observations. 

Perception is "awareness of whatever sort, however brought 
about" (Good, 1973:413). 

Product criteria are sets of observations which measure 
student gain as a method of assessing teacher effectiveness. 

Process criteria are sets of observations which measure 
teacher behaviors in order to assess teaching effectiveness. 

Presage criteria are sets of observations which measure 


teacher characteristics in order to assess teaching effectiveness. 


ASSUMPTION, DELIMITATIONS, AND LIMITATIONS 


Assumption 


The basic theoretical assumption is that perceptions of 
nursing instructors concerning evaluation of their teaching effective- 


ness are significant in that they will influence their behavior as 
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teachers. 
Delimitations of the Study 

The study was delimited to the perceptions held by 
Alberta nursing educators at one particular point in time. The 
study did not allow for the perceptions which these nursing instructors 
held regarding evaluation to be compared with those of administrators, 
students, or others who may be involved with the evaluation of 
teaching effectiveness. In addition, the study was delimited to 
those nursing instructors considered to be employed at least one 
half of a full-time position in Alberta nursing schools which prepare 
diploma or baccalaureate nurses; it did not include those instructors 
who teach in the post basic baccalaureate or master's degree programs. 
Limitations of the Study 

The study was limited by the amount of theory and research 
available on nursing instructor evaluation particularly concerning 
the aspect of product evaluation. In addition, the study instrument 
did not specify the purpose or purposes for which evaluation was 
being considered. A third limitation was that the survey instrument 
did not allow participants to make a "don't know" response to any 
section of the instrument. The instrument was constructed so that 
respondents indicated a degree of importance for each item. 

The study was limited to the thirty criteria and the twelve 
data-gathering practices outlined therein. In addition, the 
selection of criteria was not preceded by an explicit theory of 
instruction. One final limitation was that generalizations drawn 
from this study were limited to selected Alberta nursing educators. 


The presence of other variables in other populations might result in 
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Significantly different findings and conclusions if further empirical 


research were conducted. 
ORGANIZATION OF THE REMAINDER OF THE THESIS 


The following chapter deals with theory and practice related 
to the problem which has been described and delineated. Chapter III 
describes the instrument utilized in the study. It includes a 
discussion of instrument development. It also describes the sample 
of nursing instructors chosen to participate in the research, as 
well as the procedures used for collecting data. 

Chapter IV contains a description of the analysis of the 
data as well as a discussion of the findings of the data analysis. The 
final chapter of the study includes a summary, conclusions and 


implications. 
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CHAPTER II 
REVIEW OF THE RELATED LITERATURE AND RESEARCH 


Miller (1974) discussed the growing interest in the evaluation 
of teachers of higher education which is apparent in the United States. 
He indicated: 

The current interest in faculty evaluation is unprecedented 
although by no means new. Interest existed in the twenties and 
thirties, and a noticeable spurt occurred in the late forties 
and early fifties, due largely to concern about the effects of 
increased enrollment and rapid expansion of faculty on the 
quality of classroom teaching. Some falling off took place in 
the sixties, probably due to the wealth of higher education while 
expansions in program and personnel sought to keep pace with 
growth in enrollment, and due to the large infusion of federal 
and state money. Student unrest in the mid sixties and 
increasing fiscal constraints have further encouraged interest. 
The area continues to display the same vigor since about 1970, 
and continued growth in interest and programs of faculty 
evaluation can be expected tie OE 

In Miller's opinion, faculty evaluation in the 1970's was and would 
continue to be closely linked to issues of finance, governance and 
accountability. The issues cited by Miller seem to have relevance for 
the Canadian nursing education scene. Certainly funding for health 

and education seem to be only two of the current governmental priorities. 


In addition, the publication of such federal documents as A New 
Prespective on the Health of Canadians (LaLonde, 1974) may well affect 
the funding which hospitals receive. In the area of governance, Alberta 
nurse educators function under collective agreements of one sort or 
another. These agreements may place increasing emphasis on teacher 
evaluation in the future. Finally, evidence of increasing concern for 


accountability within nursing is apparent in both American and Canadian 
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literature (Poulin, 1977). 

According to Pedersen (1975) a good deal of effort has gone 
into attempting to define teaching effectiveness. He stated that the 
question: 

has enticed more researchers than any other single issue 
in education. Literally thousands of studies have been done 
that deal with such matters as characteristics of teachers, the 
effects of "good" teaching, and the goals and purposes of 
education. Yet, despite nearly half a century of rigorous 
scholarly investigation, few, if any, facts are now deemed 
established about what constitutes teacher effectiveness. In 
fact, today's researchers and educators often disagree on the 
very basis of assessment (p. 12). 

Volk (1972:13) commented that "definitions of good teaching, 
teacher effectiveness, and teacher competence are often unsupported by 
research and are as numerous as the number of evaluators." Without 
adequate definitions, it becomes difficult to carry out the process 
of evaluating teaching. Issues arise related to the specifics of the 
evaluation process. They include concerns related to the purposes of 
evaluation, who should provide evaluative input, how evaluation data 
should be gathered, and what criteria should be utilized in evaluating 


teaching effectiveness. 
PURPOSES OF TEACHER EVALUATION 


Herman (1976) indicated that evaluation programs may serve 
a single purpose or be carried out for multiple purposes. He described 
twelve of the more evident reasons for developing a teacher evaluation 
program. The reasons ranged from improving student instruction to 
providing an opportunity to congratulate an employee on a job well done. 
Wotruba and Wright (1975) discussed the purposes of teacher evaluation 


from a decision-making perspective. According to them: 
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The evaluation of teaching effectiveness is of great 
interest to three groups of decision-makers. First are admin- 
istrators who are responsible for counselling faculty members 
and for evaluating them with respect to retention, tenure, and 
promotion. Second are the teaching faculty themselves whose 
purposes are to gain feedback on their teaching ability so that 
self improvement can be facilitated. Third are the students who, 
for whatever reasons, seek information which will help them 
select instructors and courses (p. 653). 

Glasman (1976) seemed in agreement with the purposes outlined by 
Wotruba and Wright. He added that teacher evaluation could also be 
utilized for research purposes. 

There seems agreement that the most important purpose for 
teacher evaluation should be to improve teaching. Schweer and Gebbie 
(1976:184), two nurse-educators, contended that "the primary purpose 
Of appraising is that of improving teaching in terms of the total 
profession." Miller (1974:8) commented that “the overriding purpose 
of faculty, administration, and institutional evaluation must be to 
improve the instructional program." 

Proponents of teacher evaluation see that the process can 
accomplish many goals. For example, Carlson and Mable (1976: 33) 
indicated that "the evaluation process has good potential for opening 
communication and providing proper feedback." Others see that such 
positive effects are not always obtained. Pine and Boy (1975) 
recognized that negative attitudes toward evaluation exist and suggested 
that such attitudes are often related to situations where evaluation 
criteria are imposed upon teachers, or where the criteria are utilized 
without being adapted to the particular situation. Authors such as 
Carlson and Mable (1976) presented suggestions as to how evaluation 


practices might be improved. One recommendation was that the purpose of 


the evaluation be stated and communicated as clearly as possible. 
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MacKay (1974) made similar suggestions for nursing instructor evaluation 
practices. 

There seems agreement that properly conducted teacher 
evaluations can lead to improved teaching. There seems to be much 
less certainty, however, as to whether or not evaluative practices 
carried out to improve teaching should also be utilized for administra- 
tive decisions related to retention, promotion and tenure of faculty 
members. Salek (1975) advocated a separation of the two procedures 
while Eble (1970:16) suggested that it is probably "difficult and 
inadvisable to separate the two." Miller (1974:8) recommended that 
"the results of faculty evaluation should be used to assist in decisions 
about promotion and tenure... . Current economic realities make these 
decisions even more important." The issue facing the administrator 
would seem to be one of obtaining the most valid data for decision 


making from the most suitable source or sources. 
WHO SHOULD PROVIDE EVALUATIVE INPUT? 


According to Parker (1975), supervisor, self, student, and 
peer appraisal were commonly utilized practices for evaluating teaching 
effectiveness in the general education setting. Conley (1973:367), a 
nurse educator, advocated the same practices for the assessment of 
nursing instructors stating: 

Many people are in a position to evaluate the teachers 

formally or informally and should be involved in the evaluation 
process. Several types of ratings are practiced, such as 1) self 


ratings 2) peer ratings 3) student ratings 4) supervisor ratings 
and 5) expert ratings. 


Conley did not expand upon the type of individual that she saw as 


appropriate to provide expert ratings. The American Nurses’ Association 
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(1975:19, 25) published the following standard for the faculty of both 
diploma and baccalaureate nursing programs: "Faculty utilize multiple 
means for the evaluation of their continual growth as faculty members, 
i.e. self evaluation, peer evaluation, student evaluation." Recently 
the National League for Nursing Department of Diploma Programs (1977:41) 
suggested the following: 

Since the teacher is directly associated with the effective- 
ness of teaching, a self-assessment of attitudes, abilities and 
knowledge can provide the individual instructors with an 
identification of their strengths and weaknesses. Formal and 
informal evaluation by administration, peers, and students in both 
the classroom and clinical laboratory can provide clues to the 
performance of the teacher. 

As did Conley, the League recognized both formal and informal processes 
as being important for the process of nursing instructor evaluation. 
Schweer and Gebbie (1976) discussed evaluation of the nursing instructor 
in the clinical setting. They suggested that self, peer, and student 
evaluation techniques were useful. They apparently did not see super- 
visors or administrators as being appropriate to evaluate the clinical 
instructor. 

Self, peer and student evaluation techniques appeared to be 

the most popular ones among nurse educators. They seemed to see 
evaluation as a complex process and, like many general educators 


(Marks, 1976; Wolansky, 1976), saw more than one type of individual as 


being appropriate to provide evaluative input. 


Evaluation by Administrators and Supervisors 
Support for evaluation by administrators and supervisors. 


Wolansky (1976) argued that the administrator had an important role 


in teacher evaluation. He stated: 
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The administrator should be concerned and involved in the 
evaluation Process of faculty. If individual faculty member 
deficiencies are uncovered, frequently the administrator is in 
a position to provide avenues for inservice, recommend a shift 
in responsibilities to areas of strengths, or other appropriate 
decisions. Procedures to help faculty grow and develop, and 
means to carry out recommendations can be facilitated by 
administrators most effectively (p. 83). 

The National League for Nursing Department of Baccalaureate and Higher 
Degree Programs (1977:9) has set as one of its appraisal criteria 

that "the administrator of the school of nursing, with the participation 
of the faculty, is responsible for faculty appointment and review." 

Concerns about evaluation by administrators and supervisors. 
Metzer (1970) cited research which indicated that the principal's 
rating of a teacher may induce teacher conformity and create a tendency 
to rate teachers without reference to student change. 

Manatt, Palmer, and Hidlebaugh (1976:21) described the position 
of the principal in relation to teacher evaluation as follows: 

Evaluation of teacher performance is not an easy task. 
However, regulations, administrative directives and--in several 
states--statutes require that teachers be evaluated... . The 
principal finds himself in a difficult position. He is required 
to perform the conflicting functions of helping his teachers teach 
and of evaluating that performance. 

McAfee (1975) discussed a study in which teachers and their 
immediate supervisors evaluated various aspects of the teacher's 
performance. He found a wide variation between the responses of 
teachers and supervisory responses and stated, "It seems possible that 
either the teachers or the supervisors or both are incapable of correctly 
evaluating the teacher's performance, background or abilities" (p. 339). 
He suggested training in evaluation for teachers and supervisors was 
needed. 

Hayes (1976) expressed concern for the present nursing 


instructor evaluation procedures. She questioned the inclusiveness of 
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the annual employee evaluation by asking: 
Is it limited to the administrator's impression? Is the 
evaluation tool meaningful? Is there some means of peer review? 
Is there input from the peer population within nursing service 
who must work with the faculty members in the clinical setting? 
Is there on-site supervision of the faculty member's performance 
in the process of educating the student? (p. 46). 
Hayes apparently saw direct observation of performance and a variety 
of sources of peer input as being essential for effective evaluation. 
Schweer and Gebbie (1976) questioned the administrator's ability to 
be concerned with improvement of teaching. They proposed that fior 
clinical instructors: 

A more equitable evaluation can be made by a colleague 

who is studying similar teaching techniques and patterns and is 
interested in teaching improvement as opposed to evaluation by 
administrative personnel, who can evaluate faculty members for 
the purposes of promotion without real regard for the improvement 
of instructional programs (p. 184). 

The administrator seems to be in an interesting position in 
terms of teacher evaluation. Since he is responsible for the operation 
of the school and the necessary administrative decision-making involved, 
he must ensure that the process of evaluation occurs. Determining how 
and at what point in the evaluation process to become involved would 
seem to be the challenge. 

Peer Evaluation 

Support for peer evaluation. General educators such as 
McCarter (1974) and Wolansky (1976) supported peer evaluation. Peer 
review also has been advocated for some time in the United States as 
a method of evaluating the practice of nursing (Maas, Specht, and Jacox, 
1976). Hauser (1975:2204) indicated: 

At the 1972 convention, the American Nurse's Association set, 

as one of its priorities, the promotion of peer review as a method 


of maintaining standards of care. Two years later the ANA Congress 
for Nursing Practice issued "Guidelines for Peer Review". At the 
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1974 convention, ANA resolutions again emphasized standards of 
care. 


Peer review is seen as having potential for encouraging professionalism 
through the promotion of autonomy and accountability. 

It can be anticipated that educators in United States 
baccalaureate nursing programs will make increasing use of peer 
review in the future since the National League for Nursing (N.L.N.) 
Department of Baccalaureate and Higher Dearee Programs (1977:11) recently 
published criteria for accreditation which included the "participation 
in peer evaluation of teaching effectiveness" as one faculty respon- 
sibility. The criteria will come into effect in the fall of 1978 
and will most likely have an influence upon the emphasis given to 
peer review by teachers in Canadian baccalaureate and higher degree 
nursing programs. 

Although peer evaluation is one method of teacher evaluation 
which receives support in the nursing education literature, little 
information seems to be available on how the reviews should be conducted. 
The publication in the fall of 1977 of the papers from the recent 
National League for Nursing Workshop on Evaluation of Teaching Effective- 
ness (Raff, 1977; Boyle, 1977; Stone, 1977) will add to the information 
available on the subject. 

Concerns related to peer evaluation. Several years ago 
Brighton (1965:22) indicated the following disadvantages to the use 


of peer review: 


First, even when sufficient time was allowed in teachers’ 
schedules to free them to visit classrooms and make firsthand 
observations, teacher-evaluators were often reluctant to evaluate 
their colleagues’ performance formally. Second, teachers varied 
in their ability to judge personality and performance formally. 
Third a protective you-rate-me-high-and-I-wil1l-do-the-same- for-you 
attitude may develop among the corps. 
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In a recent article, Centra (1975:328) indicated that little 
“is actually known about the basis of colleague evaluations of 
instruction." Centra conducted a study in which peers evaluated college 
teachers by observing their instruction in the classroom. He found in 
his study that the teacher effectiveness ratings made by colleagues 
were generally less reliable than the student ratings of the same 
faculty members. He cautioned that consideration be given to what 
colleagues "might best judge and for what purposes" (p. 336). He 
stated that: 

Colleague ratings of teacher effectiveness based primarily 

on classroom decisions would in most instances not be reliable 
enough to use in making decisions on tenure and promotion--at 
least not without faculty members investing much more time in 
visitation or in teaching sessions (p. 335). 

It would appear that the issues related to peer evaluation 
are concerned with the facilitation of the necessary time, skills and 
attitudes to ensure that effective evaluation can occur. 

Self Evaluation 

Support for self evaluation. Wolansky (1976) advocated 
teacher self evaluation. He stated, "Self evaluation ensures the 
process of sustained motivation for self improvement" (p. 83). 
Heidgerken (1965:669) encouraged use of self evaluation by teachers of 
nursing, indicating, “Introspective analyses of one's teaching practices 
are useful whether or not other evaluative activities are undertaken." 
She provided an extensive tool for self evaluation which referred 
specifically to the selection of learning experiences. Schweer and 
Gebbie (1976) believed that the tool developed by Heidgerken could be 
adapted for use in the evaluation of clinical instruction. They 
also supported the use of "televised recordings, videotapes, audiotapes 


and other devices" (p. 184) in having clinical instructors assess their 
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own performance. 

Concerns related to self evaluation. 80]ton (1973:117) 
expressed two concerns related to teacher self evaluation. He indicated 
teachers may have difficulty analyzing specific aspects of their teaching 
if they lacked a conceptual framework or systematic observation 
system; secondly they may not have the skill necessary to operate 
audio or video equipment to record their behavior. 

Howsam, cited by Parker (1976:38), expressed the following 
concern: 

| If self-improvement were the goal, then self-evaluation 
might have considerable merit. If administrative action were to 
be based on the rating, it was doubtful whether self ratinas 
could be accepted; the ratee became both judge and jury, 
plaintiff, and defendant of his own trial. 

Several authors advocated that self evaluation be combined 
with other teacher evaluation methods. For example, Pine and Boy (1975) 
saw that such an approach utilized the teacher's internal frame of 
reference as well as capitalizing upon the external frame of reference 
which others provided. Glasman (1976) recognized that an instructor's 
need to change could be based upon a discrepancy which he recognized 
himself, or which was due to input from others who influenced him within 
the institution. 

It would seem that those who support self evaluation combined 


with other methods do so on the basis of encouraging both feedback and 


accountability. 


Evaluation by Students 
Support for student evaluation. The general education 


literature contains a good deal of information concerning the issue of 


student involvement in evaluating teaching effectiveness. Miller 
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(1974:2) stated that "those who oppose use of students' appraisals 

deny the single most important data base for judging teaching effective- 
ness." Authors such as Eble (1970), Hildebrand (1972) and Frey (1976) 
seemed in agreement that students do have a valuable contribution to 
make in this area. 

Jacobson (1966) advocated student involvement in the 
evaluation of nursing instructors. She saw nursing students as being 
particularly suited for this process. 

From the time they enter the nursing program they are 

taught to observe and record human behavior in precise terms, 
although their proficiency in this type of observation may 
vary with their level in the program (p. 219). 
Nursing students are taught assessment as a vital part of the nursing 
process. In addition, the theoretical and practical aspects of 
their performance as students are assessed on a consistent basis. Butler 
and Geitgey (1970:56) commented that nurse educators spend "much 
time and effort constructing and perfecting tools which enable us 
to judge behaviors of a student at specific levels of her education." 

Lowery, Keanne, and Hyman (1971:438) found that the under- 
graduate and graduate nursing students and faculty in their study agreed 
that students should evaluate teachers and that "length of time as a 
student or teacher seemed to have little effect on this opinion." 
Schweer and Gebbie (1976) advocated student evaluation of the clinical 
instructor. They indicated: 

Properly obtained evaluations of teaching effectiveness 

result in: 1) a means of teacher self-improvement leading 

to changes in teaching approaches and course objectives; 

2) strengthening student-teacher relationships ; 3) renewed | 
recognition of the student as an individual and as the basic 
reason for the existence of the educational institution, and 


4) greater commitment in teaching (p. 183). 


Jackson (1977) found that the diploma nursing students in her study were 
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strongly in favor of evaluating their nursing instructors in both 
the clinical and classroom settings. 

The National League of Nursing Department of Baccalaureate 
and Higher Degree Programs (1977:10) included the following in their 
accreditation criteria: 

Although ultimate responsibility for and decisions about 

the development and conduct of the education program(s) in nursing 
rest with the faculty and nurse administrator, channels are 
provided for student involvement in... . evaluation of 
teaching effectiveness. 
Student evaluation of teaching effectiveness was one method discussed 
at length at the recent N.L.N. Workshop on Evaluation of Teaching 
Effectiveness conducted for teachers in baccalaureate and higher level 
nursing programs (Raff, 1977; Stone, 1977). 

Dixon and Koerner (1976:300) made the following comment 
concerning research into student evaluation of teaching effectiveness: 
"The central research issue has become the development of tools and 
constructs to maximize the reliability and validity of this evaluation 
methodology, rather than the continued questioning of the use of such 
a methodology." 

Concerns related to student evaluation. Mims (1970:53) 
described a concern related to student evaluation: 

Many faculty members are convinced that students can make 
sound judgments and offer many useful suggestions for improving 
instruction. The issue of whether student ratings should be 
used as a basis for decision on course offerings and faculty 
promotion or tenure has not been settled. 

Concerns about the validity and reliability of student ratings inevitably 


arise when such uses of student evaluation are being considered. Centra 


(1976:335) described the situation in this way: 
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Numerous studies of student ratings over the years have 
affirmed their reliability, but their validity has been 
disputed. In particular, their relationship with how much 
students have learned in a course has been a critical question. 

Sheehan (1975) saw the issue of validity as being particularly 

important when student input was being used for administrative decision- 
making. He cautioned: 

Administrative evaluation judgments are usually irreversible 
and positions and even careers are at stake as they are made. Thus 
the information on which such decisions are made must be of 
Proven validity; and.if student ratings are to be a source of 
this information they must be able to reflect effective instruction 
(p. 688). 

He explored four factors which could cause student ratings to be invalid. 
Briefly, these factors were that student ratings of instructors did 

not always distinguish between teachers who facilitate high and low levels 
of student achievement; that rating scales were generally composed of 
evaluation items that did not tap all dimensions of teaching uniformly; 
that rating scales did not take advantage of differences in student 
learning styles; and finally, that evidence existed to show that student 
responses on rating scales could be affected by such factors as 
instructor influence tactics. Sheehan also listed factors which have 
been shown to cause variations in student ratings. He included student 
sex, student class, student age, student grade point average, subject 
matter area, class size, elected or required course, sex of instructor 
and academic rank of instructor. 

It would seem that there is considerable consensus in the 

general and nursing education literature that student input can be a 
valuable means for aiding the teacher to improve his or her teaching 


effectiveness. The agreement appears to be less when student input 


is being considered for use in administrative decision-making. Questions 
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such as the weighting which student input should be given in the 
assessment process seem relatively unexplored. 

Each source of evalutive input seems to have particular 
advantages and disadvantages associated with it, making the use of 
multiple sources of input a possible way to deal with the complex 


process of evaluation. 
DATA COLLECTION PRACTICES 


Parker (1975:109-110) indicated that the following data 
collection practices were commonly used in teacher evaluation within 
the general education setting: classroom observations, tests, 
interaction analysis, microteaching, pupil gain, questionnaires, 
inventories, anecdotal records, checklists, rating scales, open end 
statements and projective techniques. He found that the teachers in 
his study were of the opinion that classroom observation was the most 
effective data collection technique. The teachers also gave some 
Support to the use of interaction analysis, pupil gain, questionnaires; 
anecdotal records and checklists. He found "slightly more disagreement 
than agreement by classroom teachers regarding practices of collecting 
data to appraise teacher performance by tests, microteaching, 
inventories, rating scales, open end statements and projective techniques" 
(p. 87-88). 

Bolton (1972) distinguished between direct and indirect 
measures of teaching behavior. He described direct measurement as: 

The assessment of teacher performance as he attempts to 

influence learner development within the instructional setting. 
This form of assessment employs measures ranging from highly 
systematic techniques (e.g., regulated observations by trained 


observers) to less systematic techniques (e.g., casual obser- 
vations by untrained observers) and self evaluation techniques 


(Opel 2-113). 
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He described indirect measurement of teacher behavior as the assessment 
of out-of-classroom behavior such as activities in organizations, 
extra-class activities, and contributions to. curriculum development. 
Performance Observation and Recording 
It would seem that performance observation could be classified 
as a direct measure of teaching which could range from being highly 
systematic to being less systematic in nature. Observation of the 
nursing instructor could take place in the classroom, laboratory 
and patient care settings, since the nursing instructor oftén has 
responsibilities for teaching in more than one of these areas. Although 
performance observation is advocated in the nursing education literature, 
little seems to be written concerning how or by whom the process should 
be carried out. Some information on observation of teaching performance 
in the classroom is available from the general education literature. 
Concerns are raised about present clasroom observation practices. For 
example, Diamond (1975:29) discussed observation by administrators: 
Although classroom teachers receive visits from some 
administrators during the academic year, most of the observations 
merely determine the status of the teacher--should he remain on 
the job in the coming year, haw should he be rated for 
administrative purposes, should he be awarded tenure? 
Such visits are all too often brief and infrequent and what 
is generally observed by the principal or some surrogate are 
class management as well as order and responsibility for 
teaching what has been mandated by the school district or 
curriculum department. 
Miller (1974) saw classroom visitation as being a colleague 
responsibility. Eble (1970) indicated some dissatisfaction with the 
present way in which observations by colleagues were conducted. He 


stated, "Though individual faculty members in many colleges may visit 


another professor's classes, few colleges have formal boards of visitors 
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or a tradition which makes visiting classes a natural, casual and 
revealing activity" (p. 11). Centra (1975) suggested that the 
reliability of colleague assessment of teaching effectiveness based 
on classroom observation could be improved by training colleagues 
in evaluative techniques and by increasing the number of peers who 
evaluate a particular teacher's performance. 

Authors such as Frey (1976) saw the student as having an 
important role in observing teaching performance. He indicated, 
"Because students are the only regular observers in the college 
classrooms, reports of their classroom experiences provide unique 
information about the teacher and the learning environment" (p. 327). 
Other authors such as Sheehan (1975), saw concerns related to 
student evaluative input. It would seem that whether classroom obser- 
vation is carried out by students, peers or administrators, questions 
arise concerning evaluative skills and the appropriateness of 
specific evaluative tools. 

Flander's Interaction Analysis is an example of a systematic, 
direct measure of teaching effectiveness. The Flander's system of 
interaction analysis is a technique used to analyze the verbal 
interactions between teachers and students in classrooms. Bolton (1973) 
considered the Flander's Interaction Analysis system to be a single- 
factor observational system in that it is intended to focus upon one 
aspect of a teacher's behavior at a time. Anderson, Ball, and Murphy 
(1975) expressed some optimism concerning the use of tools such as 
Flander's Interaction Analysis. They indicated: 

Traditionally the effects of a teaching situation or a 

training program have been measured without observing the 


teaching process or, at best, with only enough observation 
to make a rating of the teacher's ability. With the advent 
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of new observation instruments based on systematic recording 

of observable behaviors, a more precise data-based orientation 

toward the teaching-learning process is possible (p. 266). 
On the other hand, McNeil and Popham (1973) contended that such 
instruments were not intended for teacher evaluation. They stated, 
"Examination of these instruments leads us to conclude that the 
tools are most useful for describing the teaching act rather than 
identifying instructional variables of great power or for judging 
effective teachers" (p. 220). 

Parker (1975) considered that the use of anecdotal] recording 
was a powerful adjunct to observation. He described anecdotal 
records as “a recording of progress, changes and improvements made 
as the teacher gains experience" (p. 111). There is a likelihood 
that teachers of nursing would be familiar with the process of anec- 
dotal recording since the approach has been advocated for some time 
as a method of evaluating nursing students (Rines, 1963). 
Microteaching 

DeTornyay (1971:5), a nurse educator, described micro- 
teaching as follows: 

The term microteaching as a training technique is derived 
from its characteristics--small seaments taught to a few 
students for a short period of time, in order for the teacher 
to practice a single component of teaching. Microteaching 
provides the teacher with a setting in which to practice with- 
out the complexities and responsibilities of the classroom or 
clinical setting. 

Good (1973) indicated that microteaching has been utilized in the 
preparation of teachers as well as for the inservice training of 
experienced teachers. It would seem that microteaching would have a 


definite use when the purpose for teacher evaluation was instructional 


improvement. 
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Inventories, Checklists and Rating Scales 


Parker (1975:27) described an inventory as follows: 

An inventory attempts to list as many related statements 
as possible about some teaching area. The evaluator then 
judges the extent and frequency these statements apply to the 
teaching process under observation. Inventories have value 
in directing attention to problem areas. 

A checklist can be considered an adaptation of the 
inventory. Good (1973) indicated that a checklist is a prepared list 
of items which deals with the teacher qualities, techniques, or 
conditions to be observed in a teaching situation. The checklist can 
be used by an observer for recording or appraisal, or by the teacher 
as a tool for self improvement. Schweer and Gebbie (1976) recommended 
the use of the checklist as a self improvement guide for nursing 
instructors. 

Anderson, Ball, and Murphy (1975:315) described ratings as 
"subjective assessments made on an established scale." They indicated 
that rating scales were in very common use for evaluating job behavior. 
These scales also seemed to be a popular method of obtaining input for 
research on nursing instructor evaluation (Alexander, 1968; Mims, 1970; 
Butler and Geitgey, 1970). 

Bolton (1973:115) expressed the concer that rating scales 
may be prone to error since, like other non-systematic techniques, 

"the observer, trained or otherwise, withholds his judgement until the 
end of the behavior sequence." Anderson et. al. (1975) indicated that 
the setting in which the rating took place was an important consideration. 
They saw it aS necessary that the person being rated have an opportunity 


to perform the activities on which he was being assessed. By the 


same token, the rater must have an opportunity to observe these 


edd YD mrsiroie: & ine 6 elenea enone 
dali butadhg 4 at dep Maoely 1a watentont (eced ) pene er 
\ Pre | 

ve .gougtnees yzotehleup iorsess stg tr tw ‘isda eat sr 


al 

nt dyMabdtoe st? onakrietredhttossr'e nh bav/aedo ad of avpte th wal 
“artnet GAY Yt 10, eereque <a sntieained wT vONNMAEO. He rd bs! y oO. 
Ress eel 

retusa. (Nee!) stlase: bad waswntae dronviengit Vive at ae 


ghia xGY Shhin Inomwvettm) 19a 6 ee tetlyped: rset 


2 OF 
7 


zy apatdisy badinventh (ELEsEURT | lg bie 4 TEE noite pied 
water bob vodt “:eface bafietfdsses we sm Shao ztromneeeen testa 
dotvedad doll cabTeOrRVe ve) sep iitruion: Wau ah een zation quivers ot 

ab? tuqnt printatde to vodeam yelugey © . ad of Hinaoe ofa eoleoz sont 7 
gt amie 19007 ven nienath) rio bd un Vee roeapaaeee: wit ati Ao ne o A 
—RONRE 0 198 v8 

aateoe piven dady 7 a ant veterans aint es e., 

eHeuptinines ottumadiaye- min “(saber oat -dante ces, ~ 
at Tsay taomsplut, ate soit wees 0 seated : ~ 
dade began toni “Pater! sto ite anes anataunaie 08 


pe nuit rghparene se se 28099 ios at 


ut - 7 


ee. 


behaviors. They indicated that many of the sources of error associated 
with rating scales could be avoided by the appropriate selection 
and training of the raters. 

Bolton (1973) indicated three measures in addition to 
training raters which could be utilized to make rating scales and other 
non-systematic techniques more effective. These included clearly 
defining the focus of the evaluation, developing specific, low 
inference items and using a common record form. McNeil and Popham 
(1973:232) seriously questioned the use of rating scales, indicating 
that some of the limitations associated with rating scales “make 
them of doubtful worth in the hands of administrators, supervisors 
and peers." Anderson et. al. (1975:316-317) seemed to take a more lenient 
view by suggesting that "although there are problems involved in using 
such rating forms, they should be used by trained observers when 
more objective methods of measuring performance are not available." 
Ques tionnaires 

Anderson et. al. (1975:311) described a questionnaire as: 

A group of printed questions used to elicit information 

from respondents by means of self-report. The question may be 
open-ended, requiring respondents to answer in their own words, 
or fixed choice, requiring respondents to select one or more 
answers from among those provided. The respondents may also 
be provided with checklists or rating scales. Questions may 
be concerned with the respondent's personal background, factual 
knowledge, or attitudes and opinions. 
Questionnaires have been utilized in general education research to 
obtain input regarding various aspects of teaching effectiveness 
(Rogers, 1970; Cooper, 1972; Volk, 1972). The technique has also been 
popular for research to identify effective nursing instructor 


behaviors as well as in the evaluation of nursing instructor performance 


(Layton, 1976; Armington, Reinikka and Creighton, 1972; Jackson, 1977). 
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For example, Layton (1969) used a simple questionnaire to ascertain 
which nursing instructor attitudes and actions were seen as helpful 
or harmful by two year nursing program students. 

Anderson et. al. (1975) described several weaknesses which 
may be associated with use of the questionnaire. These weaknesses 
included a low response rate, the fact that responses may be influenced 
by situationally induced differences and a concern for the self-report 
nature of the questionnaire. 

Measuring Student Gain 

Parker (1975) indicated that standardized tests were 
often utilized in research which measured student gain as a method 
of assessing teacher effectiveness. He stated: 

With this method, certain areas to be tested were 

selected (usually in skills or subject matter knowledge). 

Tests then were given and the results treated in one of 

the following ways: class standing, raw gain, achievement 

quotients, or residual gain (p. 44). 
Parker listed several limitations to use the pupil gain method. 
One such limitation was the problem of ascertaining the contribution 
made by the individual teacher. This may be a well justified concern 
when considering nursing instructor evaluation since some schools 
utilize team teaching and integrated curricular approaches. In 
addition, teachers may instruct more than one level of student so that 
the effect of the individual teacher may indeed be difficult to 
determine. For example, several instructors may have contributed to 
the student's understanding of medical nursing, which is one of the 
subject areas tested on Canadian registration examinations. 


Parker also noted imperfections in testing as a limitation 


to the use of pupil gain criteria. It may be especially difficult to 
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develop standardtzed tests to measure knowledge, skills and attitudes 
for a subject as complex as nursing. Further concerns related to 
measurement of student gains as a teacher assessment technique will 
be consideredin the discussion concerning product criteria. 

Teacher Tests and Projective Tests 

Parker (1975) indicated that standardized tests could be 
utilized in teacher evaluation. He indicated that such tests have 
been employed primarily for research purposes. He cited the National 
Teacher Examination as an example of a test which has been used 
in the United States to predict teaching effectiveness. Little 
evidence existed for the use of standardized teacher tests for 
instructors of nursing. 

McNeil and Popham (1973) believed that teacher tests could be 
utilized to distinguish among teachers. They referred to "perform- 
ance tests" or "teaching power tests." The method utilized was 
that teachers were given one or more identical objectives and a 
sample of how student gain would be measured. The objectives could 
be cognitive, affective and psychomotor in nature. The methods of 
instruction were left entirely up to the teacher; tests were administered 
following instruction to determine pupil attainment of the objectives. 
They cited several studies and stated: 

These studies suggest the conclusion that when there is 
reasonable control for extraneous factors (teacher familiarity 
with content and pupil populations) some teachers are con- 
sistently more successful than others in getting desired 
results. There is, however, need for verifying that teachers 
who can produce desired effects under conditions of teaching 
performance tests maintain their effect over time and in the 


presence of a greater range of conditions such as exist in 
conventional classrooms (p. 236). 
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Projective tests represent a different testing approach for 
assessing teaching effectiveness. Anderson et. al. (1975) indicated 
that projective tests provided an individual with an ambiguous or 
unstructured stimulus. The individual created his own responses 
which were "then interpreted as projections of the person's thoughts 
or feelings" (p. 294). They indicated that projective tests generally 
assess personality variables and noted that two of the most commonly 
utilized tests as being the Rorschach and the Thematic Apperception 
Test. 

Anderson et. al. (1975:295) cautioned against the utilization 
of these tests in any evaluative context, indicating: 

The administration and scoring of projective tests usually 
require a considerable amount of training and a great deal of 
time. Since these tests usually have to be individually 
administered and scored by a highly skilled person, the cost is 
high. If a projective test is used in evaluation, it is advis- 
able to use other personality measures (e.g., inventories, 
questionnaires) as additional means of assessing the variable 
of interest. 

Schweer and Gebbie (1976) suggested utilization of one projective 
technique for obtaining student input concerning nursing instructor 
effectiveness. They suggested the use of "forms asking students to 
complete sentences best describing their feelings or complete drawings 
to illustrate their thoughts regarding a particular incident" (p. 183). 
Otherwise, there is little evidence for the use of projective tests 

in the evaluation of nursing instructors. It is likely however, that 


nursing instructors would be familar with the technique of projective 


testing as it is utilized in patient assessment. 
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CRITERIA FOR EVALUATING TEACHING EFFECTIVENESS 


The major focus in research on nursing instructor effective- 
ness has been on criteria identification. A variety of criteria have 
been identified. 

Barnham (1965) utilized a critical incident technique to 
determine instructor behaviors which two-year college nurses thought 
to be essential. The teacher behaviors of showing restraint so that 
one's own anxiety does not influence the situation and explaining for 
Student comprehension were identified most frequently by the 
respondents. 

Jacobson (1966) utilized a modified critical incident 
technique to evolve fifty-eight critical requirements for effective 
nursing instruction. The major categories anenaded availability to 
students, ability as nurse and teacher, skill in interpersonal 
relationships, teaching practices, and evaluation practices. She 
indicated that, except for the items which dealt directly with patient 
care, the criteria could be utilized in any teaching setting. 

Alexander (1968), a Canadian nurse educator, focused on 
teaching behaviors of the instructor in the clinical setting. She 
developed a tool to assess the instructor teaching in this area and 
stated, "Separate tools are necessary for the assessment of teaching 
effectiveness in classroom instruction and clinical instruction" (p. 5). 
In making her point, she noted that almost two-thirds of the critical 
incidents identified in the Barnham (1965) study occurred while students 
were in the clinical setting. Alexander developed a 30 item Rating 


Scale for Clinical Instruction. She noted that baccalaureate program 
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instructors in her study were rated highest on the following items-- 
concern for patient welfare, interest in subject and personal 
appearance. She expressed concern for the low ratings which the 
instructors received on the items which dealt with presentation of 
subject matter and stimulation of intellectual curiosity. 

Layton (1969:27) found that in her study conducted with 
two year nursing students, "by far the most frequently mentioned 
behaviors were those that demonstrate interest in and acceptance of 
the student as a person." 

Butler and Geitgey (1970) described a tool developed by 
Butler for rating teaching effectiveness. Unlike Alexander (1968), 
they advocated that, with slight modification, the tool could be 
utilized in both the classroom and clinical settings. 

Mims (1970) constructed a trial instrument for obtaining 
student and faculty input concerning evaluation of nursing instructors. 
The items considered most important by the student respondents 
included fairness in making and grading tests, ability to interest 
students in the subject, and the systematic organization of subject 
matter. 

Lowery et. al. (1971) utilized a factor analysis approach to 
determine aspects of teaching effectiveness considered important by 
the undergraduate and graduate nursing students and faculty in their 
study. The four factors identified included concern for the inter- 
personal element in teaching practices--in particular, teacher-student 


communication; the teacher's personal warmth; ability to inspire the 


Student; and, finally, the teacher's thorough knowledge of the subject 
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matter. 

Wood (1971) utilized a portion of the rating scale developed 
by Alexander (1968) to determine the teaching effectiveness of 
several English tutors (classroom instructors) as perceived by their 
Students. Characteristics which were rated highest included the 
teacher's personal appearance, concern for patient's welfare, 
interest in subject, and knowledge of nursing practice. Wood 
considered the criteria to be in three categories--knowledge and 
skill, personal behavior, and relationships. She concluded that the 
English tutors rated high on knowledge and skill but low on personality. 
As in Alexander's study, Wood also found that tutors ranked low 
on presentation of subject matter and stimulation of intellectual 
curiosity. The high ranking of concern for patient welfare among 
classroom instructors was a particularly interesting finding. 

Armington et. al. (1972) utilized a questionnaire to have 
students rate instructors who had theory or theory and clinical instruc- 
tion responsibilities. Instructors who were ranked above average 
were those who "were enthusiastic about their work, impressed students 
as being experts in their field, encouraged students to think and 
were easily accessible to them" (p. 791). They reported that the 
characteristics of teacher effectiveness identified in their study 
tended to be similarto those of several other studies which had been 
conducted in general education. 

Kiker (1973) focused on criteria perceived as effective 
by different types of students. Kiker grouped the twelve character- 
istics into three categories--professional competence, relationships 


with students, and individual personal attributes. She found 
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significant differences between undergraduate students in nursing 
and education and graduate nursing students. The undergraduates 
rated professional competence highest while the graduate students 
rated creativity first. The study may have implications for 
nursing instructors who teach at both the graduate and under- 
graduate levels. 

Dixon and Koerner (1976) utilized a three stage study to 
identify effective nurse-educator classroom teaching behaviors. 
Two constructs were identified. Factor I was labelled individualized 
prescriptive approach. Dixon and Koerner reported that items that 
scored highest on this factor were--evaluates the students in a 
variety of ways based on objectives of course; keeps student apprised 
of his progress; and, identifies student's strengths and guides the 
student toward further development. Factor II was labelled systematic 
theoretical orientation. Items which scored highest were--demon- 
strates logical thinking process to work through complicated problems, 
highlights significant concepts and principles, and presents content 
systematically and clearly. From their study Dixon and Koerner advised 
that “investigators who tackle these issues in the future would do 
well to focus on differences between classroom and clinical teaching 
in nursing. Previous studies in the nursing literature have not tended 
to differentiate BeeNeen these aspects" (p. 305). 

Jackson (1977) conducted a study with diploma nursing students 
to ascertain their opinions concerning items which could be utilized 
for course and instructor evaluation in both the clinical and classroom 


settings. Unlike Dixon and Koerner, Jackson (171:12) concluded: 
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An instrument composed of mutually agreed upon statements 
should encourage the greatest possible degree of objectivity. 
Using one accepted form throughout the school, all students 
would be considering the same characteristics and traits for 
every instructor and course evaluated. 
A variety of criteria were identified in nursing research 
as being indicative of nursing instructor effectiveness. Most frequently 
mentioned were those which dealt with the instructor's evaluative, 
teaching and interpersonal relationship skills as well as with her 
availability to the student. 
The Mitzel Categorization of Criteria 

Mitzel (1960) proposed that criteria utilized for evaluating 
teaching effectiveness could be placed into three categories--those of 
product, process and presage criteria. According to Mitzel, product 
criteria involved a measurement of change in student behavior. He 
stated that process criteria "are most often described and measured 
in the classroom in terms of conditions, climates, or typical situations 
involving the social interactions of students and teacher" (p. 1483). 
The final category, the presage criteria, were described by Mitzel as 
"pseudo-criteria, for their relevance depends upon an assumed or 
conjectured relationship to other criteria, either process or product" 
(p. 1484). In discussing presage criteria, Mitzel further indicated: 

There are at least four types of presage variables in 
common use as criteria in teacher effectiveness research: 

a) teacher personality attributes, b) characteristics of 
teachers in training, c) teacher knowledge and achievement and 
d) inservice teacher status characteristics (p. 1484). 

Mitzel indicated that the relationship between process and 

product criteria must be emphasized. He stated, "If certain definable 


and observable educational means are clearly better than others, then 


their effects should be discoverable in measured educational ends" 
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(p. 1484). 
Recent research utilizing Mitzel's categorization of 
criteria has occurred in general education. Moore (1966), Thomas 
(1969), and Rogers (1970) used Mitzel's categorization of criteria 
in examining how principals and inspectors evaluate teachers. Moore 
identified thirty criteria for his study instrument--ten each of 
the product, process, and presage categories. Cooper (1972) modified 
the criteria developed by Moore and utilized the resultant tool to 
ascertain practices for evaluating community college personnel in 
Western Canada. Volk (1972) utilized the criteria developed by 
Moore to study the perceptions of urban Saskatchewan school teachers 
concerning actual and preferred criteria for the evaluation of their 
teaching effectiveness. No evidence was found to indicate that the 
Mitzel categorization has been employed in nursing instructor evaluation 
research. 
Product Criteria 
Support for the utilization of product criteria. The concept 
of evaluating teachers in terms of gains made by their students is not 
a new one. Parker (1975) indicated that the idea of appraising teachers 
by the achievements made by their students was advanced in the United 
States as early as 1913. Johnson et. al. (1975:178) explained the 
interest in product evaluation as follows: 
Perhaps the most obvious approach to the evaluation of 
teaching is by looking at student learning outcomes as 
direct results. The ease with which this approach can be 
stated, and its apparent common sense, no doubt largely 
account for its perennial attractiveness. 
McNeil and Popham (1973) seemed among the strongest 


present day proponents for the utilization of product criteria. They 
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stated that "a focus on pupils reveals far more about the effectiveness 
of teachers than does the direct study of teachers themselves" (p. 218) 
and advocated the pupil gain approach for research on teaching effective- 
ness. They indicated that many researchers have "succumbed" to the 
difficulties associated with this approach, 

and have opted to use more readily available criteria. By 

Studying certain procedures employed by teachers and assuming 

that these procedures are related to pupil growth, the investigator 
gets at a readily accessible process criterion and hopes it 
reflects an outcome criterion (p. 220). 

Volk (1972) found that while current Saskatchewan teacher 
evaluation practices indicated that most emphasis was given to process 
criteria and the least to product criteria, teachers preferred that 
process criteria remain of greatest importance but that increasing 
emphasis be given to product criteria. 

McNeil and Popham (1973) advocated two new approaches to 
teacher evaluation based on student gain. The concept of teacher 
power tests was discussed earlier. In addition, they suggested the use 
of contract plans which are based on student gain. They indicated that: 

the essence of this technique involves the development of 

a carefully selected set of objectives for the pupil. Super- 
visors and teachers agree in advance what they will accept as evi- 
dence that the teacher has been successful in changing the skills, 
competencies or attitudes of his students (p. 234). 

Concerns related to the utilization of product criteria. Costin 
et. al. (1971:520) described the utilizationof student gain criteria as 
being "fraught with practical and technical difficulties." McNeil and 
Popham (1973), although advocates of the approach, identified certain 


philosophical and technical difficulties associated with it. They 


discussed the technical difficulties as including: 
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concerns about the adequacy of measures for assessing a 
wide range of pupil attitudes and achievement at different 
educational levels and in diverse subject-matter areas, 
failure to account for instructional variables that the teacher 
does not control and unreliability in the results of teacher 
behavior, that is, inconsistent progress of pupils under the 
same teacher (p. 218). 

The development of adequate measures to assess teaching effectiveness 
in terms of pupil gain may be difficult in a subject such as nursing 
which has a heavy focus upon the knowlege, skill and attitude domains. 
Perhaps this is the reason that althoughproduct criteria sometimes 
receive’ support in discussions, they do not seem to have been 
incorporated into nursing instructor evaluation research or practices. 
For example, Conley (1973:367) stated that "ideally teaching success 
should be judged primarily in terms of changed behavior in students." 
She then proceeded to discuss a teacher evaluation approach which 
appeared to use process and presage criteria exclusively. 

Johnson et. al. (1975a) expressed a concern in addition to 
that of the technical difficulties associated with evaluation by student 
gain. They emphasized that teaching and learning were two separate 
processes and that whether or not learning occurs is often a matter 
of choice and/or ability on the part of the student. 
| Johnson (1975) added a third concern related to the utilization 
of product criteria. He noted that United States legislation on teacher 
evaluation has been utilizing student gain as its focus. He also seemed 
concerned with the extent to which evaluation procedures are prescribed. 
As an example, Johnson (1975:607-608) cited the California Stull Act of 
some years ago which mandated a: 

"uniform system of evaluation and assessment of performance 


of certified personnel within each school district of the state." 
Each district was ordered to develop “objective evaluation and 
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assessment guidelines" based on "standards of expected student 
progress in each area of study and techniques for the process." 
It was required to review "competence" as it "relates to 
established standards of student progress." 


Pedersen (1975) noted that with the Canadian public's increasing demand 
for accountability, product oriented teacher evaluation legislation 
may become a reality for Canadian education in the future. 


Process Criteria 


Support for the utilization of process criteria. Glasman 


(1976) saw the approach of utilizing process criteria as a valuable 
one for present day educational administrators. He stated: 


Because there are no comprehensive theories of teaching, 
one cannot be sufficiently certain about what good and bad 
teaching means and, therefore, cannot specify with certainty 
criteria for its evaluation. In the absence of such theories 
to guide the construction of appropriate evaluation instruments, 
the administrator is faced with two "strongly advocated" 
positions on the matter of instruments. A’ state legislator or 
a taxpayer wants instructors to be evaluated in terms of the 
extent to which students learn. They are essentially demanding 
‘product" evaluation. Instructors who reluctantly agree to be 
evaluated want "input" variables to be taken into account. 
They argue that students' level of intelligence or administrative 
support of instructors do effect teaching success. As researchers 
develop their theories of teaching and its subsequent evaluation, 
the administrator's most viable option at this point is to 
adopt a compromise between the two positions (pp. 314-315). 


The use of process criteria seemed to be a popular one in general 

education. Volk (1972) found that the Urban Saskatchewan school 

teachers in his study perceived that the greatest emphasis in evaluating 
their teaching effectiveness was on process criteria. He stated that his 
findings coincided with those of Moore (1966), Thomas (1969) and Rogers (1970) 
who found that "both inspectors and principals emphasized process 

criteria and de-emphasized product criteria in evaluating teachers" 

(p. 95).. Volk also found that the instructor preferences were for 
process-oriented evaluation criteria. 


The nursing studies cited earlier seemed to emphasize process 
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criteria. Interpersonal relationship skills and specific teaching 
skills seemed to be mentioned frequently. 


Concermms related to the use of process criteria. Although 


the use of process criteria for teacher evaluation has been popular, 

a major concern has been expressed regarding the fact that teaching 
behaviors may be evaluated without adequate consideration of the 
relationship of those teaching behaviors to student learning (McNeil 

and Popham, 1972). The lack of focus on this relationship in the 

past may be related to the present day emphasis on student gain criteria 
in United States educational legislation (Glasman, 1976; Pedersen, 1975; 
Johnson, 1975). 

Another concern related to the use of process criteria 
indicated that it may be difficult to distinguish between process and 
presage criteria. According to Johnson et. al. (1975a:173): 

Analysis of pedagogical behaviors fails to distinguish 

critical teaching acts from more general teacher characteristics 
interpretable in terms of teacher personality. 


Johnson et. al. (1975a) also expressed a concern related to 


the method of selecting process criteria for use in teacher evaluation. 
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They stated that the problem is with "the logical, empirical and theoretical 


grounds for the choice of any particular set of pedagogical behaviors 
as the basis for evaluation of teaching" (p. 189). They indicated 
that educational researchers in the past have very seldom presented the 


logic behind their selection of particular sets of teaching behaviors. 


Presage Criteria 
Support for the utilization of presage criteria. Johnson et. 


al.(1975a) recognized measurement of teacher characteristics as one of 


the three basic approaches to teacher evaluation. They indicated: 
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This approach to evaluation of teaching attempts to show 
that teachers with certain characteristics (such as friendliness, 
fairness, humor, sensitivity, enthusiasm, or the appearance of 
competence, for example) are approved, valued, or accepted by 
individual students or groups of students. The efficacy of 
the approach appears to rest upon the notion that learning will 
be increased if students come to perceive their teachers as 
attractive human beings. Thus, teachers who possess the supposedly 
desirable characteristics will presumably be good teachers. 
Furthermore, teachers who possess more of them, or possess them 
to a greater degree, or appear to possess them in the eyes of 
a greater number of observers, will be better teachers than those 
who have them only to a lesser degree (p. 184). 
Cooper (1972) found that when college teachers were being assessed for 
administrative promotion, presage criteria were considered as most 
important, while process criteria were stressed in the evaluation of 
instructor competence. The three presage criteria considered most 
important for administrative promotion were the degree of cooperation 
by the instructor with other staff members, the dependability of 
the instructor, and the qualities of leadership displayed by the 
instructor. Although in Cooper's research no overall difference 
existed between criteria used in the two evaluative situations, evidence 
that the importance given to specific criteria varied with the evaluative 
purpose was apparent in her study. 
Some presage criteria were identified in nursing research 
on teaching effectiveness. The teacher's personal warmth (Lowery et. al., 
1971), her personal appearance (Woods, 1971), and expertise (Armington 


et. al., 1972) were examples of the criteria identified in the nursing 


Studies. 

Concerns related to the use of presage criteria. Costin 
et. al. (1971) expressed concern over the use of presage criteria for 
evaluating teaching effectiveness. They stated that: 


although one is apt to assume intuitively that students' 
ratings of college teachers' performance should be influenced 
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by or correlated with personality traits ... . very little 

evidence exists to demonstrate whether or not this is so 

(p. 184). 
Johnson et. al. (1975a) indicated that in using teacher characteristics 
to determine effectiveness it was difficult to develop criteria 
specific to teachers. 

It seemed that the use of each of Mitzel's categories of 
criteria received support in the literature. Each approach had 
limitations associated with it. The approach or approaches selected 
for instructor evaluation may well vary with such factors as purpose, 


ease of assessment and system prescriptions which may limit the choice 


process. 
SUMMARY 


In this chapter theoretical and research literature were 
reviewed in an attempt to provide and develop the background for the 
study. The literature suggested that peer, student and self evaluative 


input were most popular for nursing instructor evaluation, while 


administrative and supervisory input was accepted for use with instructors 


not teaching in the clinical practice area. One study indicated that 
length of time as teachers did not affect the preference of various 
nursing faculty members for utilizing student evaluative input. 

The literature also suggested that a range of data gathering 
practices were utilized for teacher evaluation in general education, 
while the evaluation of nursing instructors seemed more limited to the 
use of rating scales, checklists and questionnaires. There was no 
indication as to whether or.not nursing instructors would prefer a 


broader range of data gathering practices to be used. 
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The literature seemed to indicate that most research in 
nursing education had been related to teaching effectiveness criteria. 
One study indicated that the teacher characteristics identified as 
effective in that study were similar to those identified in general 
education studies. The literature also suggested that nurse educators 
differed in opinion as to whether criteria utilized for instructor 
evaluation in a setting such as the classroom could also be utilized 
or adapted for use in evaluating the nursing instructor in other 
settings. 

No evidence was found in the literature to indicate that 
Mitzel's categorization of criteria had been utilized in nursing 
education research on teacher effectiveness. General education studies 
which used the categorization described by Mitzel determined that 
teacher evaluation practices presently emphasized process criteria. 

One study which asked teachers to indicate both existing and preferred 
criteria found that the teachers' selection of criteria varied in those 
two situations. No similarstudy seemed to have been conducted within 
Hureing education. Therefore, little seemed to be known about 

variables which might have an effect upon nursing instructor perceptions 
concerning the actual and preferred situations for evaluating their 
teaching effectiveness. For example, whether an instructor teaches in 

a baccalaureate or diploma nursing program may affect her perceptions 
concerning teacher evaluation. Similarly, with increasing educational 
preparation, a nursing instructor may be exposed to more information on 


teacher evaluation. 
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CHAPTER III 
INSTRUMENTATION AND METHODOLOGY 


This chapter provides a description of the instrument used 
in the study and of the methods used to collect and analyze data. 
The initial section deals with the procedures utilized to develop the 
personal and professional data, the criteria, and the comment aspects 
of the instrument. The final section provides a description of the 


methods employed to collect and analyze the data of this study. 


INSTRUMENTATION 


The Pilot Study 


For the purposes of this study, it was necessary to collect 
data regarding the perceptions of nursing instructors concerming 
evaluation of their teaching effectiveness. Mitzel's categorization 
of criteria was adopted as the basis for developing an instrument 
for data collection. 

Forty-two pilot study questionnaire items, related to the 
general and nursing education literature and based upon Mitzel's 
criteria as well as studies conducted by Cooper (1972) and Volk (1972), 
were constructed. The items developed were intended to represent 
the product, process and presage categories equally and were placed in 
random order in the pilot study questionnaire. Since the nursing 
education literature reviewed made little reference to product oriented 


criteria, the explanatory letter accompanying the pilot study questionnaire 
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indicated that criteria which measured nursing instructor effectiveness 
in terms of student performance had been included in the questionnaire. 

The pilot study questionnaire was circulated to thirty 
nursing instructors in Alberta and British Columbia known to be teaching 
in diploma or basic baccalaureate nursing programs, or to have done so 
within the last three years. In Section I of the instrument, pilot 
study subjects were asked to provide specific personal and professional 
data. 

Section II of the questionnaire requested pilot study subjects 
to rate criteria in terms of their perception of the importance which 
each criterion was given (actual) and should be given (preferred) in 
the evaluation of their teaching effectiveness. The following rating 
Scale was utilized: 

0 Don't know 

1 None or very limited importance 
2 Some importance 

3 Moderate importance 

4 Great importance 

5 Very great importance 

In Section III of the questionnaire, respondents were asked to 
work through the 42 items a second time and to rate each item on an 
eight-point scale in terms of its clarity. A rating of one meant 
that the item was unclear, while eight indicated that the meaning of 
the item was clear. In addition, respondents were requested to make 
suggestions in the space provided below each item as to how the clarity 


of the item might be improved. 
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In Section IV, subjects were requested to comment on the 
questionnaire or on the topic of nursing instructor evaluation in general. 

Responses were received from 27 (90 percent) of the pilot 
study subjects. Difficulties which the pilot study subjects had in 
completing Section I of the instrument were noted so that these problems 
could be avoided in the construction of the Nursing Instructor Evaluation 
Instrument. 

The responses to Sections II and III made by 25 pilot study 
participants (83 percent) were analyzed by computer. Two additional 
questionnaires were received following analysis. Comments made on the 
various sections of these instruments were noted. 

Frequencies and means were calculated for the actual and 
preferred responses to each item in Section II in order to determine 
the importance given to each item by the pilot study participants. These 
results aided in the selection of criteria for inclusion in the Nursing 
Instructor Evaluation Instrument. 

The results of Section II of the pilot study were factor 
analyzed by use of a varimax rotation procedure in order to determine if 
the criteria represented product, process and presage categorizations 
as proposed by Mitzel. Five, four, three and two factor solutions were 
attempted for both the actual and preferred series of responses. Each 
of the factor analyses on the preferred responses accounted for more of 
the total variance than each of the comparable analyses on the actual 
responses. For that reason, the results of the preferred analysis were 
utilized. The three factor solution seemed to represent the best 
distribution of significant loadings. Table 1 shows the three factor 


solution for the preferred criteria. Factor I seemed to deal with product 
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Table | 


Pilot Study Factor Solution for Preferred 
Criteria Emerging From Varimax 
Rotation and a Three Factor Solution 


Items Communalities Factor I Factor II Factor III 
ree Re SA 1 RRS ik AE odes re ee 
1 0.278 -0.481 -0.042 alle 

0.596 0.719 0.091 -0.264 
3. 0. 383 -0.18] -0.159 0.596 
4. -0.618 -0.18) OR? 0.756 
oe 6.159 0.045 0.379 0.116 
6. 0. 456 0.295 0.605 -0.053 
Ws 05313 -0.952 Weel) 0.155 
8. 0.435 0.180 0.605 0.189 
9. 0.624 0.716 0.057 0.329 
10, 0.392 -0.016 0.624 -0.040 
Laie 0.551 -0.197 0.716 -9.021 
ize 0.294 -0. 343 0.419 0.040 
is 0.441] 0.103 0.583 0.301 
14. 0.683 0.208 0.789 -0.127 
Ty 0.719 0.806 0.117 Or20/ 
16. 0.504 0.396 -0.011 0.589 
Vis 0.346 0.088 QO. 32! 0.485 
18. 0.496 0.049 0.082 0.698 
19, 0.691 0.008 0.800 Qiao 
20. 0.225 -0.162 0.404 0.191 


= ——— 
ee ee 


c1a0 | ety. . fab.0- ; ats, 7 . 
98. i | to. 4, a 
aeg,0 palo. taf a a 
We.) vt 12D aft 
tro ce.) Reb eer. 
cane 229.0 a eo 
ay. —Ct(<“<‘ai‘ézr oe; 0- li) RB cats 
wet. , a0 par ao oS 
eb 0 ta med a os 
nbt).0- sa.0 ait Se 
fsa,0- ae) MLD (eee, 

d,0 3 da yn 
lop fio oh 

To kb age ERAS pati 
wes,0 Te: Wt 
wo Ts a a he 


Table 1 (continued) 


Items Communalities ‘Factor I Factor IT Factor III 
ape 0.584 0.176 0.727 0.156 
pon 0.735 0.852 ~0.095 0.024 
23 0.455 0.671 0.067 0.019 
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25. 0. 396 0.593 0.175 -0.118 
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34. 0.797 0.191 0.355 0.797 
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39. 0.173 0.096 0.350 0.202 
40. 0.601 0.675 0.148 0.352 
41, 0.527 -0.104 0. 390 0.604 
42, 0.355 -9.094 0.588 -0.002 


Percentage of Common Variance 
100.000 30.798 32.381 28. 821 
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criteria, Factor II with items of a process nature and Factor III 

with the presage criteria. The results of the three factor solution. 
were utilized to identify items which could be included in the Nursing 
Instructor Evaluation Instrument to represent product, process and 
presage criteria. 

Frequencies and means were calculated to determine the 
perceived clarity of each of the 42 criteria included in Section III 
of the instrument. The clarity ratings and the extensive suggestions 
for item improvement that were made by pilot study participants aided 
in the selection and modification of items for inclusion in the 
Nursing Instructor Evaluation Instrument. 

Comments made on Section IV of the pilot study instrument 
were categorized according to: general comments concerning evaluation 
of nursing instructors, comments concerning instrument construction 
and comments concerning criteria for evaluating teaching effectiveness. 
Out of the 27 questionnaires returned, 19 (70 percent) contained comments 
in Section IV. Seventy-four percent of those who completed Section IV 
made at least one comment in their responses concerning the criteria 
which dealt with student gain. The high response rate and the nature 
of the responses to this section indicated that subjects completing 
the Nursing Instructor Evaluation Instrument might make use of a similar 
section if it were included in that instrument. 

The Nursing Instructor Evaluaton Instrument 

The Nursing Instructor Evaluation Instrument is included in 
Appendix A, page 148. It contains three sections. Section I, Personal 
and Professional Data, includes the following nine variables: age, 


highest level of education, amount of teaching experience, length of 
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employment, type of present employment, hours employed at present, 
area of major teaching responsibility, type of program and approximate 
number of full-time faculty. It was thought that these variables 
might have considerable relevance to the perceptions being studied. 
Section II of the Nursing Instructor Evaluation Instrument 
deals with evaluators, data gathering practices and criteria for 
evaluating nursing instructors. The same rating scale was utilized 
throughout Section II. The response key was as follows: 
1 Very limited importance 
2 Limited importance 
3 Moderate importance 
4 Great importance 
5 Very great importance 
Section II(a) of the Nursing Instructor Evaluation Instrument 
asks respondents for their perceptions concerning the importance which 
specific types of individuals had (actual) and should have (preferred) 
in the evaluation of nursing instructors. Types of individuals were 
utilized rather than specific titles since nursing education programs 
were known to have a variety of administrative structures. Space 
was provided for respondents to specify other personnel who might be 
involved in evaluating their teaching effectiveness and to rate their 
perceived importance. 
Section II(b) deals with practices which might be undertaken 


to gather information for evaluating nursing instructors. A brief 


description of each practice was included since the pilot study suggested 


that extensive systems of nursing instructor evaluation might not be 


common. The descriptions included-were a modified version of those 
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utilized by Parker (1976). 

Section II(c) deals with criteria for evaluating teaching 
effectiveness. Ten criteria from each of the process, presage and 
product categories as identified by the pilot study factor analysis 
were retained for the questionniare. The criteria selected were 
modified considerably from the pilot study based upon input from the 
pilot study subjects. Modifications generally took the form of 
making an item more specific or rewriting a criterion in behavioral 
terms. 

A table of random numbers was utilized to determine the 
order which the categories would take throughout Section II(c). Within 
the categories, items were arranged in what was perceived to be 
an easy to difficulty ordering according to epee estes of the factor 
analysis, the assessment of the clarity of each item by the 
pilot study respondents, and efforts to ensurethat product items were 
preceeded and followed by criteria which clearly referred to the 
instructor. 

Section III encouraged respondents to comment on nursing 


instructor evaluation in general and/or the questionnaire itself. 
METHODOLOGY 


Collection of Data 

The Nursing Instructor Evaluation Instrument was distributed 
to nurse educators who were teaching in Alberta diploma or basic 
baccalaureate nursing programs at the time of the study. In order to 
obtain participants for the study, a request was made to meet with 


the administrators who formed the Consortium of Senior Nurse Educators. 
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The administrators were informed of the nature of the proposed research 
on nursing instructor evaluation at their January 1977 meeting. A 
follow-up letter was then sent to each Consortium member, asking that 
person to provide a list of faculty members whom they considered to 

be teaching nursing one-half time or more in their program. They 

were requested to include both those employed sessionally and on a 
permanent basis. In addition, the letter requested permission to 

send the questionnaire and a follow-up letter to the work address of 
each faculty member provided. Faculty lists were subsequently received 
for all diploma and basic baccalaureate programs in Alberta. 

Two hundred and thirty one questionnaires were mailed to the 
nursing instructors identified for the study. The questionnaire included 
a covering letter which indicated that the anonymity of individual 
responses would be ensured. A self-addressed, stamped envelope was 
enclosed for returning the questionnaire. In addition, a self-addressed, 
stamped post card was included for those who wished to receive a summary 
of the study results. A reminder post card was sent to all participants 
ten days after the original questionnaires had been distributed. A 
follow-up letter was distributed approximately one month following the 
reminder post card in order to encourage any remaining study participants 
who wished to complete the questionnaire to do so as soon as possible. 
All correspondence is included in Appendix B, page 158. 

One hundred and eighty nine questionnaires were returned 
Of these, five were not usable, leaving one hundred and eighty four 
returns to be analyzed. The usable returns represented eighty percent of 


the study participants. 
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Statistical Analyses 


Pearson and Phi correlational procedures were utilized to 
determine the relationships between the nine personal and professional 
variables included in the study. 

In analyzing the data from Section II of the questionnaire, 
Standard deviations and rankings of means were utilized to show the 
extent of common perceptions concerning the importance of actual and 
preferred evaluators, data gathering practices and criteria. 

The Spearman rho correlation from ranks was used to determine 
the degree of relationship between the importance given to the various 
evaluators, data gathering practices and criteria in the actual and 
preferred situations. 

T tests were utilized to determine whether differences which 


existed in perceptions concerning actual and preferred evaluators, 


data gathering practices and criteria were of statistical significance. 


T and F tests were used to determine the effect which the 


nine personal and professional variables had upon perceptions concerning 


actual and preferred evaluators and data gathering practices. 


A factor analysis was done to determine if criteria tended 


ay 


to cluster in Mitzel's categories of process, product and presage criteria. 


T and F tests were utilized to determine the effects which 
the nine personal and professional variables had upon preferences for 


product, process and presage criteria. 
SUMMARY 


The data for this study were collected by the use of a 


questionnaire. In addition to questions dealing with personal and 
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professional information, and questionnaire included-measures for 

types of evaluators, data gathering practices and criteria for assessing 
nursing instructor effectiveness. All Alberta nursing instructors 
teaching one half time or more in basic baccalaureate or diploma 


‘nursing schools were included in this study. 
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CHAPTER IV 
ANALYSIS OF THE DATA AND DISCUSSION OF 
THE FINDINGS 


This chapter provides a description of the findings which 
emerged when different types of analyses were applied to the data, as 
well as a discussion of those findings. The initial section deals with 
preliminary findings regarding personal and professional variables as 
well as relationships between the independent variables. Findings 
related to types of evaluators, data gathering practices and criteria 
are discussed in subsequent sections. The final section presents a 


summary of the chapter. 
PRELIMINARY FINDINGS 


Personal and Professional Information 
The nursing instructors were asked to answer nine ques tions 
presented under the heading, Personal and Professional Data. Information 
was obtained concerning the instructor's age, level of education, 
amount of teaching experience, length of present employment, type of 
present employment, hours of present employment, area of major teaching 
responsibility, type of program and number of full-time faculty. 
Frequency and percentage distributions were used to summarize 
the responses to the nine questions. Table 2 presents information 
concerning the respondents. Since there were infrequent responses to 


some of the items, the data collected from this section of the 
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questionnaire were collapsed into the categories indicated on the 
table. 

Table 2 indicates that 60.4 percent of the instructors were 
between the ages of 26 and 35. Table 2 also indicates that 4 out of 5 
- Of the instructors had a baccalaureate degree or a baccalaureate plus 
some additional preparation while only 9.2 percent of the instructors 
were prepared at the master's level. The relatively small number of 
nurses prepared at the master's level indicates that this study was 
concerned primarily with perceptions concerning nursing instructor 
evaluation as held by instructors with baccalaureate preparation. Studies 
involving more master's level prepared nurses might produce findings 
significantly different from the ones emerging in this study. 

Table 2 shows that approximately one half of the nursing 
instructors had less than 5 years of teaching experience; however, a 
full 81 percent of the instructors had been employed in their present 
teaching positions for less than 3 years. The employment mobility 
which characterized the study participants might have provided them 
with an understanding of a variety of institutional practices which 
existed for nursing instructor evaluation; on the other hand, teachers 
who were new to an institution might be uncertain as to what evaluation 
practices did exist. Table 2 indicates that most of the nursing 
instructors were employed on a permanent basis and were working in a 


full-time capacity. Table 2 also indicates that about two-thirds of 


nursing instructors had approximately equal clinical and classroom teaching 


responsibilities. The fact that the majority of respondents had teaching 
responsibilities in both areas is likely to have had an effect upon 


their responses in terms of selection of evaluators, data gathering 
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practices and criteria considered appropriate for both settings. 

Table 2 reveals that 87 percent of the instructors taught in programs 
which prepared diploma nurses. The study primarily involved instructors 
teaching in poepaea and college based programs since the preparation 

- of diploma nurses occurs in those settings. Studies involving more 
instructors teaching in baccalaureate programs might produce 
Significantly different findings from the ones emerging in this study. 
Finally, Table 2 shows that schools with 10-19 faculty members 

were most common; however, a variety of school sizes is apparent. 
Relationships Between the Independent Variables 

Correlational procedures were utilized to explore relation- 
ships between the independent variables. All possible relationships 
between the 9 variables are reported in Table 3. The table shows 
that 32 out of a possible 36 relationships were found to be 
statistically significant. 

Table 3 shows that approximately two-thirds of the significant 
relationships were positive. Many of the positive relationships such 
as the correlation between age and amount of teaching experience were 
expected; however, one relationship of note is that which occurred 
between age and number of faculty. The finding indicates that older 
instructors tended to be employed at the larger schools. Table 3 
also shows that older teachers and those with more education tended 
to stay in a place of employment longer. In addition, Table 3 shows 
significant relationships between length of present employment and 
school size. The finding indicates that instructors working in 
larger schools tended to have been employed longer than those teaching 


in the smaller schools. 


ehots aistt at thee no ih doer | 
Sipdneni uriuok @fO6 Atl atoodse! aait ainhe: © Ts 
Soevsyde oth Qe0he footie ea waltss” “at 
~oreifey siotdxe in) bss ficw eter - ladiiaied eae 
agilanpitalon efdteent SCN + ealdelted taapiaqeant em: 
wuts aldad afi 2 al de? of Dednensy ove eatdomyad © ont | 
od of bauot iow col depp heater Bf ateteteg @Vo tia 369s 
| Sate 
‘grep tiaghe ods, "Yo abr iitesout fad an xaayh 26MP works & eter = 
the aqidandttaten avidteow att to qralt autatsog. 4 
Saw oon hiagne eitito sos’ 1. Jinuome bo sgt nacMied siertw levine . 
bay Yano ote seat a? ster Yo qrlenotinien eng tivewor Gg beaaeqne 
srbsfoldeu ebaao tbat onthe alt Aa Tage 30 odoin lab es! eanied: - 7 | 
Eater: Zloeie vpNel sits Dayoigne eo on baa eve . 
babies xOLeGUbs sron rity aeDAd Hine evens isbla dan aoe RTS ve | 
airie © aTdgT .noH9 Visbe ni <ragpait Soeyations Yo-ss6lgen. At ye GE 
ae bie span it yan Simeon: tr ANaneT neues <q iMenoteatam Daag 
| |e pitsron avotoutant 60 anteater grithinth ot Aah Hebe 
| es 


= 


wan 
oe 


7 


aa 


a 


59 


LeveL Go° ze yuedLsLUbis, [eLuasig JULOg = qd 
Lara, 10° ze JUuedLYLUBLSy,y }UALILJYJ$90D LUd = O 
LOABL LOO’ 3 QUCILSLUBLSyxe A SUOSARad = J 
Aznoe4 
$90 AOQUINN 6 
yan weuboug 
xx 4025 °0 $0 adh} 8g 
’ - ALLL LGLSuodsay 
xxx4Qh°QO xxx 94dZ9°O BuLusee, uoley / 
; . : uawAo | dw 
xxx Gd€y 0 $60°0  xxxqd99°0 + Miseei 
40 S4noH 9 
euxddS°0 81°0 -xxxqd9g°0 —_xx$99°0 a cea 
$O adh] GC 
wxx41Z2°O  xx%4d82°0 ¥¥HxAE E °O xxx 4d€ °O xxx Gd9E *O Bs re 
$0 Yzbue] 7 
AZO*O-  -x¥%9d81°0 ¥x4AQL "0 *4d€1°0 *x¥4d91°O = -x¥%49H7°0 are 
$0 Junowy ¢ 
¥t4EL°O- xxxGd9Z°O0-  — xxx4972°0- Gdg0°O- x¥xqd6€°O- = x¥4GL°O x4 910 uOLzeONpy 
$0 LOAD] Z 
¥¥x46L°O xxx 9d97 0 ¥¥¥A67°O xxx GdyE°O —-xxxddlG°0 xxxtZE°O 4eetGG°O xx491‘0- aby | 
6 8 L 9 S t € % 


e__eGV\€6v—t US 


SOLZSLUa}OBULYD [PUOLSSAJOUg puke L[PUOSUag S,40}9NUYSUT BULSUNN UaaMzag SdLYSUOLzeL ay 
. € 91gel 


ae Sai etaget oce 


ee ie ings te 
>) 
ie: Se “Sig ald? a 


60 


Table 3 indicates that 8 (out of a possible 36) relation- 
ships had significant negative correlations. One relationship of 
note is that which occurred between age and level of education. The 
finding indicated a tendency for younger instructors to be more 
highly educated. The table also shows a correlation indicating that 
there was a slight tendency for those with more education to be 


employed in the smaller schools. 
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TYPES OF EVALUATORS 


Importance and Consensus of Perceptions 


One of the purposes of the study was to determine the extent 
to which Alberta nurse educators shared common perceptions as to who 
was and should be involved in providing input for evaluating their 
teaching effectiveness. 

Table 4 presents the importance which the various evaluators 
were givenin the actual and preferred situations by ranking the means 
for each situation and indicating the difference in rank between the 
two situations. Table 4 reveals that senior administrators were ranked 
as being the most important source of evaluative input, while the 
study respondents preferred that input from the instructors themselves, 
immediate supervisors and peers be of more importance than that provided 
by the senior administrator. Input from immediate supervisors received 
the same ranking in both situations (2) as did input from students (4). 
Table 4 shows that the instructor herself was perceived by the study 
participants as their most preferred source for evaluative input. The 
table shows that the Spearman rho was .49. This value indicates that ) 


the overall similarity in perceptions concerning actual and preferred 
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Table 4 


A Comparison of Perceived Importance of Actual 
and Preferred Involvement of Evaluators 


Mean Score Mean Rank Difference 


Type of Evaluator Actual Preferred Actual Preferred in Rank 


Senior administrator(s) 3.36 36 ] 4 -3 
Immediate supervisors 3028 3.94 C 2 9) 
Peer instructors 2.24 3.42 5 3 2 
Instructors themselves 2.99 4.12 3 ] 2 
Students 2.66 8.30 4 4 0 
Others 2.00 cag) 6 5 ] 


Rho = .49; Not significant 
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evaluators was not great enough to be of statistical significance. 

Table 5 presents the extent of consensus in perceptions 
concerning the importance which various evaluators were given in the 
actual and preferred situations by presenting the standard Maviet iene 
for each type of evaluator. The standard deviation was used as a 
measure of consensus since it was recognized that a large standard 
deviation would reflect a considerable dispersion or spread in a set 
of scores while a small standard deviation would reflect a set of 
scores which were less variable (Popham and Sirotnik, 1973). 

Table 5 shows that the most consensus existed concerning 
the actual use of others to provide evaluative input. In responding 
to the section of the instrument which dealt with evaluators, study 
Subjects were asked to name personnel other than administrators, 
supervisors, peers, instructors themselves and students who were or 
should be involved in providing evaluative input. The respondents were 
then requested to rate the type of individual named in terms of their 
perceptions of the actual and preferred importance of that individual 
in evaluating teaching effectiveness. Sixty-two respondents (29.7 
percent) completed the other category. Their responses were categorized 
and indicated that 62.9 percent of the subjects saw the head nurse, or 
head nurse in conjunction with the ward staff or nursing supervisor 
as being most suitable as other sources of evaluative input. Others in 
the clinical area such as the clinical nurse specialist and the 
physician were named by 14.5 percent of the respondents. The remainder 
of the subjects who completed the other category (22.6 percent) 


indicated that personnel outside of the nursing practice setting should 
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Table 5 


Extent of Common Perceptions Concerning 
Actual and Preferred Evaluators 


LE TR a I ee 
IO LL i nero 


Standard Deviation 


Type of Evaluator Actual Preferred 
Senior administrator(s) 1.38 0.95 
Immediate supervisors 1.40 0.83 
Peer instructors esl 1.04 
Instructors themselves leo? 0.88 
Students ibs ae 1.03 
Others 120 Ozog 
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provide evaluative input. Individuals such as curriculum coordinators, 
secretarial staff, registrars and campus directors were suggested, 

Table 5 indicates that perceptions concerning the actual 
use Of peers, and then students, had the next most common agreement. 
Table 5 also shows that the least most common perceptions existed 
concerning the role which immediate supervisors played in evaluating 
teaching effectiveness. 

Table 5 shows that, within the preferred situation, the 
most common agreement existed for the role which immediate supervisors 
should play in providing evaluative input. The next most common 
perceptions were for the use of instructors themselves and senior 
administrators, in that order. Table 5 also shows that the least most 
common perceptions existed concerning the role which peer instructors 
should play. 

Table 5 shows that standard deviations for each of the 
preferred evaluators were smaller than those of the actual evaluators, 
indicating that nursing instructors were more similar in their 
perceptions of who should evaluate than in their understanding of the 
existing use of evaluators. 

Differences Between Actual and Preferred Evaluators 

The study also addressed itself to the question of whether 
or not any differences found between the perceptions concerning actual 
and preferred evaluators were of statistical Significance. Table 6 
compares the actual and preferred evaluators by utilizing the T test. 
Table 6 reveals that statistically significant differences existed for 


five out of six evaluators, with the greatest difference occurring 
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Table 6 


A Comparison of Perceived Importance of Actual and 
Preferred Involvement of Evaluators by Means 


AE Aa Beene pris 
Senior administrator(s) 3. 36 3.36 05 
Immediate supervisors Shorts 3.94 =0..0 eae, 
Peer instructors 2.04 3.42 = 1 eo ON ex 
Instructors themselves 2.99 4.12 -12.04*** 
Students 2.66 3. 36 =026) se 
Others 2.00 2.91 -5.05*** 


—es 


*kkSignificant at .001 level 


between the actual and preferred use of instructors themselves 
as a source of input (T value, -12.04). Table 6 indicates that 
the difference between the actual and preferred use of peer instructors 
was almost as great (T value, -11.59). No statistically significant 
difference occurred in the actual and preferred use of senior 
administrators, but all otherevaluators took on increasing importance 
in the preferred situation, as indicated by the increase in means from 
actual to preferred. 
Effect of Independent Variables on Perceptions Concerning Evaluators 

The study was concerned with the effect which the selected 
personal and professional variables had upon the perceptions of the 
study participants concerning actual and preferred evaluators. T 
and F tests were utilized. | 

Table 7 shows the effect which type of employment had. The 
table shows that 1 out of 12 comparisons reached statistical signifiance. 
The large number of non-significant findings indicates that instructors 
employed sessionally and on a permanent basis were largely in agreement 
concerning actual and preferred evaluators. The significant difference 
which did occur was that those employed on a sessional basis had a 
stronger preference for student evaluative input than did those employed 
on a permanent basis. 

Table 8 indicates that no statistically significant differences 
(out of a possible 12) occurred between those employed 50 to 74 percent 
of full-time and those employed 75 percent of full time to full-time. 
This finding suggests that the groups were homogeneous in their 


perceptions concerning actual and preferred evaluators. 
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Table 7 


Effect of Type of Employment on Perceptions of 
Actual and Preferred Evaluators by Means 


Type of Employment T Value 
Type of Evaluator Sessional Permanent 
Senior administrator(s) 
ACTUAL 3.03 3.43 -1.42 
PREFERRED Saal 3.36 -0.28 
Immediate Supervisors 
ACTUAL 3.69 3.19 Vall 
PREFERRED 4.07 3.91 0.93 
Peer instructors 
ACTUAL Pays S73) 2WL2 0.64 
PREFERRED 3.59 3.39 0.92 
Instructors themselves 
ACTUAL 2.66 3.06 -1.46 
PREFERRED 3.93 4.16 -1.27 
Students 
ACTUAL 2.93 2.61 1.25 
PRE FERRED 5270 3.30 1.96* 
Others 
ACTUAL) 2200 2.00 O22) 
PRE FERRED Cail 3202 -1.02 


*Significant at .05 level 


Table 8 


Effect of Hours of Employment on Perceptions 
of Actual and Preferred Evaluators by Means 


Percentage of Full Time T Value 
Type of Evaluator 50%-74% g 75% 100% 
Senior administrator(s) 
ACTUAL 2.91 3.42 -1.64 
PREFERRED 3.09 3.38 -1.34 
Immediate supervisors 
ACTUAL Ba03 3225 0.25 
PREFERRED 4.09 3.92 0.92 
Peer instructors 
ACTUAL Delo 2.26 -0.29 
PRE FERRED a 3.43 -0.67 
Instructors themselves 
ACTUAL Pah 3.96 -0.95 
PREFERRED 3.90 Ail, -1.28 
Students 
ACTUAL 2.91 2.61 1.90 
PREFERRED 3.55 eh ot! 0.86 
Others 
ACTUAL ibeards 05 -0.79 


Die 
PREFERRED 2209 3.04 =<lal2 
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Table 9 indicates that 3 Statistically significant differences 
(out of a possible 12) occurred between instructors teaching in 
diploma and baccalaureate nursing programs. It reveals that baccalaureate 
program instructors saw more use being made of student evaluative 
‘input at present than did diploma instructors. It also indicates that 
instructors in the baccalaureate programs had a stronger preference 
for both peer and student input than did instructors teaching in diploma 
nursing programs. 

Table 10 presents the effect of age on instructor perceptions 
concerning who does and should evaluate. One finding out of 12 was 
statistically significant, indicating that respondents of various ages 
were largely in agreement concerning actual and preferred evaluators. 
The significant finding was concerning the effect of age on perceptions 
of the extent to which immediate supervisors were presently utilized. 
Differences were found between the 30 and under and over 40 age groups; 
between the 36-40 and the over 40 age groups; and between the 31-35 
and the 36-40 age group. Those instructors over 40 perceived immediate 
supervisors to be utilized to the greatest extent. They were followed 
by those 31-35, the instructor group under 30 and, finally, the 36-40 
age group. 

Table 11 indicates that no statistically significant differences 
existed between instructors with various levels of education concerning 
actual evaluators. 

Table 11 indicates that two statistically significant findings 
were related to the effect of level of education on preferences for 
student and peer evaluative input. The difference in preference for 


peer evaluation occurred between the group of instructors prepared at the 
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Table 9 


Effect of Type of Program on Perceptions of 
Actual and Preferred Evaluators by Means 


Type of Program T Value 

Type of Evaluator RN. BASEN 
Senior administrator(s) 

ACTUAL SoD 3.46 -0.35 

PREFERRED cts} Sy 1.04 
Immediate supervisors 

ACTUAL 3.24 3.48 -0.75 

PREFERRED 3.94 Saor 0.42 
Peer instructors 

ACTUAL Dues 238 -0.55 

PREFERRED BaG2 4.08 -4,5]*** 
Instructors themselves 

ACTUAL 2.98 3.08 -0.4] 

PREFERRED 4.11 Aa) -0.27 
Students 

ACTUAL 2.58 Bol, -2.05* 

PREFERRED 3425 Bley, -4,28*** 
Others 

ACTUAL 2808 1.80 0.41 

PREFERRED (apis 3.33 -0.96 


ee 


*kkSignificant at .001 level 
*Significant at .05 level 
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Table 10 


Effect of Age on Perceptions of Actual 
and Preferred Evaluators by Means 


‘Type of Evaluator £30 


Senior administrator(s) 


ACTUAL Bi | ee 

PREFERRED 3. 39 
Immediate supervisors 

ACTUAL 3.00 

PREFERRED 3.83 
Peer instructors 

ACTUAL (2 SW 

PREFERRED 3.40 
Instructors themselves 

ACTUAL 3.00 

PREFERRED 4.19 
Students 

ACTUAL 2.45 

PREFERRED 3.26 
Others 

ACTUAL 1.75 

PREFERRED 2.92 


Age in Years 
31-35 36-40 >40 


Zec0 1290 2.16 
Bear Se) Sas 
2n92 Cast Al Sree) 
4.08 Bao 4.12 
2.69 cui 3.03 
S238 302 3-00 
2.00 (ic GE) 2e00 
33.08 3210 cod! 


**kSignificant at .001 level 
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Table 11 


Effect of Level of Education on Perceptions of 
Actual and Preferred Evaluators by Means 


: Level of Education F Ratio 
Type of Evaluator R.N. Beoce Mas ters 
Senior administrator(s) 
ACTUAL 3.28 Shy ey 3. 88 1.31 
PREFERRED ee ly. a Bea 0.40 
Immediate supervisors 
ACTUAL PEG 3. 36 3,12 1.50 
PREFERRED 3.94 3.94 3.94 0.00 
Peer instructors 
ACTUAL 1.94 2225 2253 12.04 
PREFERRED 3.44 S20 4.18 5. 23** 
Instructors themselves 
ACTUAL 207 2.99 3.35 1.09 
PREFERRED 4,39 aeO 7, 4.29 1.45 
Students 
ACTUAL 2.89 2e07 3.18 1.94 
PRE FERRED 2.94 gaac 4.24 8. 38*** 
Others 
ACTUAL 1.66 2.05 2.00 Oee7 
PREFERRED 3.50 2.90 3.00 0.99 


*kkSignificant at .001 level 
**Significant at .01 level 
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master's level and the instructors prepared at the baccalaureate 
level. The group prepared at the master's level had the strongest 
preference for peer evaluation, followed by those at the R.N. level, 
and, finally, by the baccalaureate prepared group of nursing instructors. 
Table 11 shows the second statistically significant finding to be 
the effect of level of education on preference for student evaluative 
input. The difference occurred between the master's and diploma 
prepared nurses and the master's and baccalaureate prepared nurses. 
The master's level instructors had the greatest preference for 
student input, followed by the baccalaureate prepared instructors and, 
finally, the diploma prepared teachers. 

Table 12 presents the effect of amount of teaching experience 
on actual and preferred evaluators. Table 12 indicates that only 1 
out of a possible 12 differences was found to be of statistical signif- 
icance, indicating that respondents who had been teaching for various 
lengths of time held similar perceptions concerning evaluators. The 
significant difference which did occur was between those with 3-4 
years and those with 5-8 years of teaching experience. The group with 
5-8 years of teaching experience saw the most use being made of the 
immediate supervisor in present evaluation practices. They were followed 
by those with over 8 years experience, instructors with less than 3 
years and, finally, by the group with 3-4 years of experience. 

Table 13 indicates that no statistically significant differences 
occurred in the perceptions of groups employed for varying lengths of 
time concerning the actual and preferred importance of input from 


individuals other than the instructors themselves. 
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Table 12 


Effect of Amount of Teaching Experience on Perceptions 
of Actual and Preferred Evaluators by 


Means 
? Years of Teaching Experience F Ratio 
Type of Evaluator £3 3-4 5-8 58 
Senior administrator(s) 
ACTUAL 6), 740) 3820 3.54 3.67 26) 
PREFERRED Gao 3.46 3.24 hy ote} 0.43 
Immediate supervisors 
ACTUAL Salo Bete! 3.64 3.49 22 10% 
PREFERRED 32/6 3.98 4.00 4.06 10 
Peer instructors 
ACTUAL 2.40 Cee Mae: 2e35 0.68 
PREFERRED 3.42 Sa20 3.40 Sia tH) 2.06 
Instructors themselves 
ACTUAL 2.85 oe NS detail! 3.20 Ons 
PREFERRED 4.00 4.23 4.09 4.20 0.68 
Students 
ACTUAL 2.42 eo8 2.84 2.97 1.68 
PREFERRED Seg Bilis Sie) 3.60 2.36 
Others 
ACTUAL 2.00 1.82 leco 2.67 eo 
PREFERRED 2.69 Seles 2.90 2.92 Uso 


*Significant at .05 level 


aohrart enttagt Xe ies 
BS a3 ja aaa 


ate 


air 


Effect of Length of Present Employment on Perceptions of 
Actual and Preferred Evaluators by Means 


Type of Evaluator 


Senior administrator(s) 
ACTUAL 
PREFERRED 


Immediate supervisors 
ACTUAL 
PREFERRED 


Peer instructors 
ACTUAL 
PREFERRED 


Instructors themselves 
ACTUAL 
PREFERRED 


Students 
ACTUAL 
PREFERRED 


Others 


ACTUAL 
PREFERRED 


*Significant at .05 level 


Table 13 


Years of Present Employment 


<1 


w Pr Ww Po 


wr 


Jae 
ol, 


male) 
.90 


yA 
230 


.68 
ae 


Bo) 
220 


ais: 
Ae 


1-2 


WwW PO w ho w PO 


— 


. 36 
ao 


w00 
98 


43 
45 


709 
296 


. 60 
. 30 


58 
ares 


3-4 


w W 


w PO 


ww PO 


. 10 
3a 


.02 
94 


~ 10 
a/c 


4] 
say 


She 
4] 


00 
Ci) 


WwW Ww WwW 


wh 


w ho 


5+ 


eh 
47 


a00 
94 


(ead) 
. 56 


Beli 
26 


fs} 
700 


ws 
ly 


F Ratio 


[a> Bp) 


—i 


— 


—_ 


45 
24 


. 80 
a0, 


. 60 
. 30 


. 60* 
24% 


42 
42 


.68 
41 


1D 


Sica 4 


—— 


term fie 
42 


= 
i 


——— 


sigessd Yo ete 
> me) 


a 


: - 7 - . — 
nel pe anny Te 


a 


ov. 
> 


"ba 


Hu rid rn 
awfermedd a0 aurea” 
ig? % 7 - oT » Ag 
— ata deg - 


12 : , 7 ‘hc 
re be 7 aa 


Table 13 shows that groups employed for varying lengths 
of time differed significantly in their perceptions of the importance 
which input from instructors themselves had and should have. The 
Significant difference in perceptions occurred between those employed 
for less than one year and those employed between 3 and 4 years. 

Those instructors employed between 3 and 4 years gave the most 
importance of any group to the use of instructors themselves as actual 
and preferred evaluators. They were followed by those who had been 
employed 5 or more years, those employed 1-2 years and, finally, by 
those who had worked in their present employment for less than one year. 

Table 14 presents the analysis of the effect of major 
teaching responsibility on actual and preferred evaluators. The 
absence of statistically significant differences among those whose 
major teaching responsibility was clinical, classroom, or both, 
indicated homogeneity in terms of their perceptions of who did and who 
should evaluate. 

Table 15 indicates the effect of the number of faculty (school 
size) on perceptions. By far the greatest number of statistically 
significant differences between groups were apparent with this 
independent variable. (Seven out of a possible 12 were significant) . 
Table 15 reveals a difference due to school size for every type of 
actual evaluator except the other category. 

The difference in perception of the importance which senior 
administrators had in providing evaluative input shown in Table 15 
occurred between the largest schools (30-39 faculty) and the next two 


smaller schools (20-29 and 10-19 faculty members respectively). 
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Table 14 


Effect of Major Teaching Responsibility on Perceptions 
of Actual and Preferred Evaluators by Means 


Responsibility F Ratio 
Type of Evaluator Classroom Clinical Classroom/ 
Clinical 

Senior administrator(s) 

ACTUAL 330 aa20 3.45 0.60 

PREFERRED 3.30 3.45 Bao. OR / 
Immediate supervisors 

ACTUAL 2.70 3.00 3.42 2.50 

PREFERRED 4.40 3.80 3.94 Zico 
Peer instructors 

ACTUAL 2.20 2.14 2.30 On3Z 

PREFERRED 3. 30 3.14 3.55 2.80 
Instructors themselves 

ACTUAL 3.50 2.96 2.96 0472 

PREFERRED 4.50 4.06 4.10 S07. 
Students 

ACTUAL 2.90 2.69 2.63 0.21 

PRE FERRED 3.40 3.29 3.39 0.15 
Others 

ACTUAL 1.00 2eah 1.90 lel 


PREFERRED 4.00 2n05 3.10 2.49 
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Table 15 


Effect of Number of Faculty on Perceptions 
of Actual and Preferred Evaluators by Means 


Number of Full-Time Faculty F Ratio 
*RADS ONES £10 10-19 20-29 30-39 


Senior administrator(s) 


ACTUAL sa0 3.69 See 2.57 8. B4*** 

PREFERRED Bie 3.25 3.40 3.50 0.62 
Immediate supervisors 

ACTUAL 3.83 45.647 Shall 2.15 14.22*** 

PREFERRED 4423 3.91 3.94 3.74 2-03 
Peer instructors 

ACTUAL 2.97 Zee 1.86 2A33 5 .68*** 

PREFERRED 3.70 She! Sais) 3.02 6.40*** 
Instructors themselves 

ACTUAL 3.07 Salo Ny Siyate. 3.90** 

PREFERRED 4.10 4.10 4.04 4.29 0.64 
Students 

ACTUAL 3210 2.95 1.90 2.79 8. 70*** 

PREFERRED 3.40 3.73 3.06 3414 5.06** 
Others . 

ACTUAL 2.03 1.85 2.07 1.50 2243 

PREFERRED 2.88 3327 2.81 2258 1.59 


**kSignificant at .001 level 
**Significant at .01 level 
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Participants in school sizes of 20-29 saw senior administrators as 
having the greatest importance in evaluating instructors. They 
were followed by the schools of 10-19 in size, the schools of under 
(10 faculty and, finally, the largest schools (30-39 faculty). 

Table 15 shows a difference in perception among groups 
concerning the importance which immediate supervisors had in evaluating 
teaching effectiveness. The difference in perception concerning this 
importance occurred between those from the largest schools and those 
from every other school size. The smallest schools saw the most 
use being made of the immediate supervisor. They were followed by 
schools of 20-29 faculty size, 10-19 size and, finally, by the 
largest schools (30-39 faculty). 

Table 15 indicates that a difference in perception among groups 
occurred concerning the actual use of peer instructors. The difference 
Occurred between instructors from the smallest schools (under 10) 
and the next two larger schools (10-19 and 20-29 respectively). The 
smallest schools saw the most use being made of peer evaluation. They 
were followed by the largest schools, the schools with 10-19 faculty 
and, finally by schools with 20-29 faculty members. 

Table 15 shows that the statistically significant difference 
among group perceptions concerning the importance which instructors 
as evaluators were given occurred between the largest faculties (30-39) 
and the next two smaller faculty groups. The largest schools saw this 
source of input as being most important. They were followed by the 
schools of 10-19 faculty members, the smallest schools and, finally, the 


schools with 20-29 instructors. 
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Table 15 reveals that faculty members from schools of different 
sizes saw student input as being of varying importance in present 
practices. The difference occurred between faculties of 20-29 
in size and every other group. Instructors in the smallest schools saw 
‘themselves as utilizing student input to the greatest extent. They 
were followed by faculties of 10-19 in size, 30-39 in size, and, 
finally, 20-29 in size. Instructors from the smaller schools perceived 
that student input was utilized more than did instructors from the 
larger schools. 

The significant difference in preferences for peer input shown 
in Table 15 occurred between the faculty groups of largest and smallest 
size, between the largest and next to smallest group and between the 
groups of intermediate size. The faculty group of 10-19 had the 
greatest preference for peer evaluation. They were followed by the 
faculties under 10, the faculties from 20-29 in size, and finally, 
the largest faculty groups. It would appear that faculty who teach 
in the smallest schools preferred peer evaluation to the greatest 
extent. 

Table 15 indicates that a significant difference existed in 
terms of what faculty from various sized schools preferred in terms 
of student input. The difference occurred between the second smallest 
schools (10-19) and the two larger groups (20-29, and 30-39 respectively). 
The second smallest schools expressed the greatest preference for student 
input. They were followed by the smallest schools, the largest schools 
and the schools of size 20-29, in that order. A comparison of the 
actual and preferred situations in terms of student evaluative input 


indicated that the smallest schools preferred less emphasis upon student 
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input, while the second smallest schools (10-19) preferred that 
considerably more emphasis be placed on students as a source of 
input. 

The non-significant findings concerning preferred evaluators 
as Outlined in Table 15 indicate that faculties of varying sizes 
seemed to have similar preferences for the importance which senior 
administrators, supervisors, instructors themselves and others 
should have in evaluating their teaching effectiveness. The 
perceptions seemed to vary most concerning the importance which peer 


and student evaluation should be given. 


DATA GATHERING PRACTICES 


Importance and Consensus of Perceptions 


One of the purposes of the study was to determine the extent 
to which Alberta nurse educators shared common perceptions concerning 
the importance of data gathering practices which were being or 
should be used in evaluating their instructional effectiveness. 

Table 16 indicates that importance which various data 
gathering practices were given in the actual and preferred situations 
by a ranking of means and a comparison of the difference in rank between 
the two situations. Table 16 shows that teacher self appraisal was 
seen as the most common method of obtaining evaluation input. The 
table also reveals that self appraisal was the preferred method of 
choice. Student gain was the second most common method by which teacher 
assessment information was gathered; however, the study participants 
preferred that it be given much less importance. Checklists and rating 


scales rated third and fourth, respectively. 
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Table 


16 


A Comparison of Perceived Importance of Actual and 
Preferred Data Gathering Practices 


Practice 


Performance observation 
Interaction analysis 
Anecdotal records 
Microteaching 
Checklists 

Rating scales 
Inventories 

Ques tionnaires 
Student gain 
Projective tests 
Teacher tests 


Self appraisal 


Mean Score 
Actual 
1.68 3.42 
1.61 3.29 
1.68 3810 
hares 34305 
ny eet 
lez] 2.44 
130 Dac) 
Warde) 1.95 
2203 2e05 
1.03 1635 
1.14 20141 
2.39 4.07 


Mean Score 


Preferred Actual Preferred 


Rho = .70; Significant at .05 level 
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Difference 
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Table 16 indicates that the preference was for the use 
of performance observation, microteaching and interaction analysis 
in addition to self appraisal. These findings suggest that the nursing 
instructors preferred direct observation of their teaching behavior 
to be utilized in addition to self appraisal. 

Table 16 shows that the Spearman rho was .70. This 
value suggests that there was a positive correlation between the 
actual and preferred use of the data gathering practices; that is the 
data gathering practices perceived to play a major role in the existing 
situation tended to be seen as playing a major role in the preferred 
situation and, conversely, data gathering practices perceived to play a 
minor role in the existing situation tended to be seen as playing a 
minor role in the preferred situation. 

Table 17 shows the extent of consensus in perceptions concerning 
the importance which various data gathering practices were given in the 
actual and preferred situations by presenting the standard deviation 
for each data gathering practice. 

Table 17 indicates that the most agreement about existing 
practices occurred concerning the use of projective tests, teacher tests, 
questionnaires and inventories. Table 17 also shows that the 
participants shared the most differing perceptions concerning the use 
of self appraisal, performance observation and student gain. Table 17 
reveals that self appraisal had the greatest degree of variance as 
indicated by the standard deviation obtained. 

Table 17 shows that the most commonly shared perceptions were 
concerning the preferred use of projective tests, self appraisal, 


questionnaires and teacher tests, in that order. Table 17 also indicates 
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Table 17 


Extent of Common Perceptions Concerning 
Actual and Preferred Data Gathering Practices 


Standard Deviation 


Data Gathering Practice eae Berend 
Performance observation | ee UTS 110 
Interaction analysis 1.01 1.04 
Anecdotal records 1.08 1.08 
Microteaching. 0.86 1202 
Checklists 1.06 1.07 
Rating scales 08 ar 1.08 
Inventories . 0.76 1.01 
Questionnaires O06 0.91 
Student gain a6 15 
Projective tests 0723 0.69 
Teacher tests O85T 0.92 
Self appraisal 1.29 0.89 
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that the most variance in the respondents' perceptions occurred 
concerning the importance which student gain, performance observation, 
anecdotal records and rating scales should have. Measuring student 
gain was the preferred data gathering practice about which educators 
expressed the greatest difference in perception. 

Differences Between Actual and Preferred Practices 

The study was also concerned with the question of whether 
or not any differences which occurred between the actual and preferred 
data gathering practices were of statistical significance. Table 18 
indicates that a significant difference was found between the actual 
and preferred situation for each of the 12 data collection practices. 
Table 18 also shows that the means for each of the practices increased 
from the actual to the preferred situation, indicating that the study 
participants were in favor of more emphasis being placed on each 
practice. The table also shows that perceptions about microteaching, 
interaction analysis and self appraisal demonstrated the greatest 
differences between the actual and preferred situations. 

Effect of Independent Variables on Perceptions Concerning Practices 

The study also considered the effect which the selected 
independent variables might have had upon the perceptions of study 
participants concerning actual and preferred data gathering practices. 
T and F tests were utilized. 

Table 19 shows the effect of type of employment on data 
collection practices. The table indicates that 4 out of 24 comparisons 
reached statistical significance. This finding suggests that sessional 
and permanent employees were generally in agreement concerning actual 


and preferred data gathering practices. 
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A Comparison of Perceived Importance of Actual and 
Preferred Data Gathering Practices by Means 


*kkSignificant at .001 level 


Table 18 


ier ACTUAL PREFERRED ee 
Performance observation 1.68 3.42 =16.28%"* 
Interaction analysis 1.6] 3.29 -18.37*** 
Anecdotal records 1.68 336 =|5. 000" = 
Microteaching 1.36 3. 30 -20. 80*** 
checklists ee 2.79 =i) Os hes 
Rating scales al 2.44 - 8.16*** 
Inventories 1.30 2.21 -11.46*** 
Questionnaires 1.20 1295 = 05 7b02* 
Student gain 2.03 2.75 - 6.69*** 
Projective tests 1.403 1a35 - 6.34*** 
Teacher tests 1.14 ea -13.42*** 
Self appraisal 2339 4.07 -16.90*** 
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Effect of Type of Employment on Perceptions of Actual 
and Preferred Data Gathering Practices by Means 


Data Gathering Practice 


Performance observation 
ACTUAL 
PREFERRED 

Interaction analysis 
ACTUAL 
PREFERRED 

Anecdotal records 
ACTUAL 
PREFERRED 

Microteaching 
ACTUAL 
PREFERRED 

Checklists 
ACTUAL 
PREFERRED 

Rating scales 
ACTUAL 
PREFERRED 

Inventories 
ACTUAL 
PREFERRED 

Questionnaires 
ACTUAL 
PREFERRED 

Student gain 
ACTUAL 
PREFERRED 

Projective tests 
ACTUAL 
PREFERRED 

Teacher tests 
ACTUAL 
PREFERRED 

Self appraisal 
ACTUAL 
PREFERRED 


Table 19 
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Table 19 indicates those who were employed sessionally 
perceived that performance observation was utilized significantly 
more than did those who were employed on a permanent basis. Table 
19 also reveals that those employed on a permanent basis had a 
Significantly stronger preference for the use of microteaching than 
did those employed sessionally. In addition, the table shows that 
permanent employees had a stronger preference for use of projective 
tests and self-appraisal. 

Table 20 indicates the effect which hours of employment 
had upon perceptions concerning data gathering practices. The table 
shows that 2 out of 24 comparisons reached statistical significance. 
The great number of non-significant results indicates that those 
employed from 75 percent of full-time to full-time generally shared 
the same perceptions concering data collection practices for teacher 
evaluation as did those employed from 50 to 74 percent of full-time. 

Table 20 shows that those employed in a full time or near to 
full-time capacity saw ratings scalesas utilized more than did those 
employed less than 75 percent of a full-time capacity. In addition, 
those employed on a 75 percent of full-time to full-time basis 
had a stronger preference for the use of projective tests and self 
appraisal than did those employed on more of a part-time basis. 

Table 21 reveals that out of 24 comparisons between the 
perceptions of instructors teaching in diploma and baccalaureate 


programs concerning data collection practices, none were statistically 


significant. These findings indicate a high degree of agreement between 


the groups as to actual and preferred data gathering practices. 


Table 22 reports the effect which the variable of age had 
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Table 20 


Effect of Hours of Employment on Perceptions of Actual 
and Preferred Data Gathering Practices by Means 


: ; Percentage of Full-Time T Value 

Data Gathering Practice 504-744, 75%~-100% 
Performance observation 

ACTUAL 2215 1.66 1.44 

PREFERRED 3.62 sss 1.00 
Interaction analysis 

ACTUAL 1.48 1262 -0.61 

PREFERRED 3.14 BA 3d -0.73 
Anecdotal records 

ACTUAL e720 1.66 0.15 

PREFERRED Seek Se15 0.49 
Microteaching 

ACTUAL lass 1% 35 -0.11 

PREFERRED 2.90 Sesto -1.90 
Checklists 

ACTUAL 1.65 71 -).26 

PREFERRED 2.64 2.80 -0.67 
Rating scales 

ACTUAL 1235 i733 -2.40* 

PREFERRED 2.36 2.46 -0.37 
Inventories 

ACTUAL 1.20 1.28 -0.53 

PREFERRED 2.41 2.19 0.94 
Questionnaires 

ACTUAL 1.40 Ae / Wiz 

PREFERRED 2.18 1.89 1.44 
Student gain 

ACTUAL 210 2.01 0.88 

PREFERRED 2.86 213 0.48 
Projective tests 

ACTUAL 1200 1.04 -0.69 

PREFERRED 1.09 1.38 -3.40*** 
Teacher tests 

ACTUAL 1.10 19 -0.40 

PREFERRED 2.19 eel 0. 38 
Self appraisal 

ACTUAL 2.05 2.44 -1.26 

PREFERRED 3.68 4.14 -1.72** 


ee ec tt A 


*kkSignificant at .001 level 
*kSignificant at .01 level 
*Significant at .05 level 
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Table 21 


Effect of Type of Program on Perceptions of Actual 
and Preferred Data Gathering Practices by Means 


. é Type of Program T Value 

Data Gathering Practice RN. B.Sc. 
Performance observation 

ACTUAL 1473 1.35 1.44 

PREFERRED 3.46 3217 Tei22 
Interaction analysis 

ACTUAL 1.58 aoe -1.03 

PREFERRED 3232 3.08 1.03 
Anecdotal records 

ACTUAL 1470 1254 0.67 

PREFERRED genc ay fl ean 
Microteaching 

ACTUAL 1337 1-26 0.56 

PREFERRED S05 2.96 1.75 
Checklists 

ACTUAL - 1.74 1.79 -0.23 

PREFERRED 2.84 2.46 1.61 
Rating scales 

ACTUAL Tag 1.78 -0.34 

PREFERRED 2.46 2,203 OF 52 
Inventories 

ACTUAL (gas 42 -0.91 

PREFERRED 2.28 1.79 2hee 
Ques tionnai res 

ACTUAL 121 il a2 .96 

PREFERRED 2e02 1.50 2.66 
Student gain 

ACTUAL 2.09 163 1.84 

PREFERRED 2.1/5 PD -0.01 
Projective tests 

ACTUAL 1.04 1.00 0.73 

PREFERRED tha sv ‘re 1.04 
Teacher tests 

ACTUAL ‘Weal AS 0.15 

PREFERRED 2413 2.00 0.66 
Self appraisal 

ACTUAL 2.34 2.475 -1.46 

PREFERRED 4.09 3.96 0.70 
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Table 22 


Effect of Age on Perceptions of Actual and 
Preferred Data Gathering Practices by Means 


Data Gathering Practice 


Performance observation 
ACTUAL 
PREFERRED 

Interaction analysis 
ACTUAL 
PREFERRED 

Anecdotal records 
ACTUAL 
PREFERRED 

Microteaching 
ACTUAL 
PREFERRED 

Checklists 
ACTUAL 
PREFERRED 

Rating scales 
ACTUAL 
PREFERRED 

Inventories 
ACTUAL 
PREFERRED 

Questionnaires 
ACTUAL 
PREFERRED 

Student gain 
ACTUAL 
PREFERRED 

Projective tests 
ACTUAL 
PREFERRED 

Teacher tests 
ACTUAL 
PREFERRED 

Self appraisal 
ACTUAL 
PREFERRED 
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**Significant at .01 level 
*Significant at .05 level 
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upon perceptions concerning data gathering practices. The table 
indicates that age had an effect upon perceptions concerning 3 out 
of 24 data collection practices. The large number of non-significant 
findings in Table 22 indicates that nursing instructors of various 
ages were largely in agreement concerning actual and preferred 

data collection practices. 

Table 22 indicates that a significant difference in perceptions 
Occurred concerning the actual importance of performance observation. 
The difference in perceptions was between the 31-35 and the over 
40 age group. The 31-35 age group saw performance observation as 
being of most importance. They were followed by the 36-40 age group, 
then the under 30 and, finally, the over 40 age group. 

Table 22 reveals that age also had a significant effect 
upon preferences for the use of anecdotal records. The difference 
occurred between the group aged 36 to 40 and those over 40. The 
36-40 age group had the strongest preference for the use of anecdotal 
records, followed by the under 30 group, those 31-35 and, finally, 
the over 40 age group. 

Table 22 shows that age also had an effect upon preferences 
for measuring student gain as a method of gathering data conceming 
instructional effectiveness. The significant difference occurred 
between the over 40 age group and the under 30 and 31-35 age groups. 
The over 40 age group preferred the use of student gain as a data 
collection method. They were followed by the 36-40 age group, the 
under 30 and the 31-35 group respectively. The findings indicate 
that the two oldest groups of instructors were the most favorable 


toward utilization of student gain. 
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Table 23 shows that no statistically significant differences 
were found among mean perceptions of instructors with various levels 
of education conceming data collection practices. These findings 
‘indicate that the groups were highly in agreement concerning the 
actual and preferred use of the data collection techniques included 
in the study. 

Table 24 indicates the effect of the variable, amount 
of teaching experience, on perceptions concerning actual and preferred 
data collection practices. Table 24 reveals that only 3 out of 
a possible 24 statistically significant differences occurred. These 
findings indicate that except for the practices of performance 
observation and measuring student gain, instructors with varying amounts 
of experience seemed in agreement concerning how information concerning 
their teaching effectiveness should be collected. 

Table 24 shows that amount of teaching experience affected 
the perceptions held concerning the extent to which performance 
observation was being utilized as a data collection practice. The 
significant difference occurred between the group with 5 to 8 years 
of teaching experience and the group with over 8 years. Those with 
5-8 years of teaching experience saw performance observation as being 
of most importance. They were followed by those with 3-4 years of 
teaching experience, those with less than 3 years and, finally, those 
with over 8 years of teaching experience. 

Table 24 shows that amount of teaching experience had a 


significant effect upon perceptions concerning both the actual and 


preferred use of student gain. The differences in perceptions concerning 


present utilization of student gain practices occurred between the group 
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Table 23 


Effect of Level of Education on Perceptions of Actual 
and Preferred Data Gathering Practices by Means 


Se cee ee ee ER eee eee 
i a en ee ee aR a 


. . Level of Education F Ratio 

Data Gathering Practice B See serene 
Performance observation 

ACTUAL le 78 leyz2 1.29 1.02 

PREFERRED 3.29 3243 3.47 0.123 
Interaction analysis 

ACTUAL 1.6/7 1.64 1.29 0.94 

PREFERRED 3.50 6.23 3eD3 1.03 
Anecdotal records 

ACTUAL 1.44 7} lea? 0.47 

PREFERRED Ba33 3.14 3.18 0.26 
Microteaching 

ACTUAL 1.50 1233 1.44 0.39 

PREFERRED Shri BOL 3.35 0. 34 
Checklists 

ACTUAL 1250 Ai) ESZ 0.56 

PREFERRED ee OV, 2.79 2.88 0.18 
Rating scales 

ACTUAL le ic Je0 1.282 0.11 

PREFERRED 2.56 2.41 2.59 0x32 
Inventories 

ACTUAL hae le2/ 1.59 1.69 

PREFERRED 2.39 2.20 2.168 0.30 
Questionnaires 

ACTUAL 22 1220 1.18 0.03 

PREFERRED 2.06 1.97 1.65 aali2 
Student gain 

ACTUAL 2.50 Uhl 1.65 2.45 

PREFERRED 2.78 27! ZeBe 0.05 
Projective tests 

ACTUAL 1.06 ios 1.00 0.25 

PRE FERRED eee iho she! 1.18 0.98 
Teacher tests 

ACTUAL lice ihe is} ee 0.58 

PREFERRED 2820 Cole 2.00 0.41 
Self appraisal 

ACTUAL A pei e) rae she! 2.82 1.19 

PREFERRED nee ee 4.05 4.18 0.43 
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Table 24 


Effect of Amount of Teaching Experience on Perceptions 
of Actual and Preferred Data Gathering Practices by Means 


: : Years of Teaching Experience F Ratio 

Data Gathering Practice £3 3-4 5-8 38 
Performance observation 

ACTUAL 1.56 1.81 2.00 1.26 2.93* 

PRE FERRED 3.39 3.49 3.53 3..20 0.70 
Interaction analysis 

ACTUAL 1.75 1.59 1.64. 1.40 0.84 

PREFERRED 3.40 3.39 82720 3.08 0.95 
Anecdotal records 

ACTUAL 1372 1.75 1.60 1.61 0823 

PREFERRED 3.19 3.08 3.20 Sh 0:12 
Microteaching 

ACTUAL lest 35 ea 35 0.14 

PREFERRED S74 3.21 3.39 8433 0.28 
Checklists 

ACTUAL 1.75 1267, loo le74 0.18 

PREFERRED 2.69 2.88 2.67 2.91 0.59 
Rating scales 

ACTUAL View A 1.67 1.82 1.63 0.25 

PREFERRED 2025 2.63 2.43 2.44 1.05 
Inventories 

ACTUAL 1.31 1.30 2g he3i 0.04 

PREFERRED rh ee 2&3) Zale, 2220 0.35 
Ques tionnaires 

ACTUAL Wore 1.15 ear 3 eee 

PREFERRED 1.87 1.90 2.00 2.06 0.41 
Student gain 

ACTUAL Isa75 14485 2.09 2.58 4.38** 

PREFERRED 2.58 2.69 Zea Bee 3.01* 
Projective tests 

ACTUAL 1.04 102 e0V 1.00 0.56 

PREFERRED He Si 1.24 resks} 1.52 135 
Teacher tests 

ACTUAL 1.10 1.15 tal] 1.26 0.85 

PREFERRED 2.00 2.14 2.08 2.29 0.70 
Self appraisal 

ACTUAL Ziel 2.25 2/0 2.46 1.42 

PREFERRED 4.08 4.00 4313 4.11 0.20 


tenia nti 


**Significant at .01 level 
*Significant at .05 level 
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with over 8 years of teaching experience and those groups with 

under 3 years experience and between 3 and 4 years. Those with the 
greatest amount of teaching experience (over 8 years) gave the 

most importance to student gain; each less experienced group 
perceived that increasingly less emphasis was being given to student 
gain as a data collection practice. Table 24 indicates that the signi- 
ficant difference in amount of emphasis preferred for student gain 
practices occurred between those with over 8 years of instructional 
experience and the groups who had been teaching for less than 3 

and between 5 and 8 years. Those with over 8 years of experience 
had the greatest preference for the utilization of this method, 
followed by the groups with 5-8 years, 3-4 years and under 3 years, 
respectively. 

Table 25 presents data showing the effect which being 
employed in the same institution for varying lenaths of time had upon 
perceptions of study participants concerning data collection practices. 
The table indicates that a high number of non-significant relationships 
existed. It seems that nursing instructors who had been employed for 
various lengths of time expressed a high degree of agreement concerning 
actual and preferred data collection practices. The one statistically 
significant difference which occurred was between those employed 
the least length of time (less than 1 year) and thosewho had worked 
the longest (over 5 years) concerning their preference for the use of 
microteaching. Of the 4 groups, those employed over 5 years had the 
greatest preference for the use of microteaching. They were followed 
by those employed 1-2 years, those employed 3 to 4 years and, finally, by 


those with less than 1 year of experience, in that order. 
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Table 25 


Effect of Length of Present Employment on Perceptions 
of Actual and Preferred Data Gathering Practices by Means 


3 A Years of Present Employment F Ratio 

Data Gathering Practice <1 1-2 3-4 b+ 
Performance observation 

ACTUAL 1.86 1287 1.64 12 Za55 

PREFERRED gaee 3.4] 3.64 Bea] 1227, 
Interaction analysis 

ACTUAL 17 d02 iMey2 1.59 0.29 

PREFERRED Sh) 323 Buss 3.34 Oe 
Anecdotal records 

ACTUAL 1.84 ess 1.51 1.50 1.40 

PREFERRED . 3.04 Shes | 3.08 3.40 0.92 
Microteaching . 

ACTUAL 1.43 lea eeu 1.24 0.45 

PREFERRED 2.98 3530 3933 Bt03 a7 02% 
Checklists 

ACTUAL 1.86 ‘87.0 ie ite! 1e5S 0.53 

PREFERRED 2.80 Zaie 2.84 Zed6 Oel2 
Rating scales 

ACTUAL 155 a7 7 UR TEE hn TAS 0.50 

PREFERRED 2033 2610) 2.61 2.54 0.95 
Inventories 

ACTUAL LANs 1.39 1.22 1.48 1.92 

PREFERRED 2.02 pee 2.29 be, SXe) 1.05 
Questionnaires 

ACTUAL 1 e20 ies te) lioz 08/3 

PREFERRED 1.86 2307 2.08 leva les7, 
Student gain 

ACTUAL 1.80 Hotels 2e25 2.26 2 |e) 

PREFERRED 2.63 gy 2.92 2.97 1.56 
Projective tests 

ACTUAL 1.04 1.00 1.08 1.00 1.22 

PREFERRED e32 130 47 (hers) 0.80 
Teacher tests 

ACTUAL VEG lets 1310 18211 0.36 

PREFERRED 2.06 1.96 2.25 rd ay 1.03 
Self appraisal 

ACTUAL 2.10 2.36 Zod, 2. Oi 1.49 

PREFERRED 3.94 3.94 4.18 4.32 1v92 


*Significant at .05 level 
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Table 26 indicates the effect of major teaching responsibility 
upon perceptions concerning the various data collection practices. 

The table shows that only 3 out of a possible 24 differences were 
Statistically significant. These findings indicate a high degree 

of agreement among these groups concerning actual and preferred data 
collection practices. When significant differences did occur, they 
were related to the preferred situation. 

Table 26 presents a difference which occurred between 
those who taught only in the classroom and those who had equal class- 
room and clinical responsibilities; the highest preference for the 
use of questionnaires occurred with those who taught primarily in the 
classroom, followed by those whose major responsibility was in the 
clinical area, and finally, with those whose responsibilities included 
both classroom and clinical instruction. 

Table 26 indicates that a significant difference occurred 
concerning the preference of the groups for the use of projective tests. 
The classroom instructors were found to be different from both 
other groups in this preference. Classroom teachers most strongly 
favored the use of projective tests, followed by those who had equal 
classroom and clinical responsibilities and, finally, by those who 
instructed in the clinical areas. Table 26 indicates that a third 
difference in perceptions occurred between the classroom instructors 
and each other group concerning preferences for the use of teacher tests. 
Classroom teachers had the greatest preference for the use of teacher 
tests, followed by those with equal clinical and classroom responsibilities 
and, finally, by those who instructed primarily in the clinical area. 


Table 27 presents the effect of number of full-time faculty 


he <* Le 


eal) 
‘ s “. 


rere fonts) vot | a : 
peering attest sate “ic 
oraw sonata 1 we si saedy o to ud ea 
owvnab tigts 8 sind thet apirontt seat mon 

ateh hoaryne fine. tnutes @ntersato> Zor, im me 
ould . Yeas bt essnneth ls dasort Hote nai. 20a}. nt 
| Nong ayt te ‘tune nee of b a - 
nesaten beware A> irn onnarett tb 8 zd ngzeg: a ataet “ 


' 
~ ghee, Le 
-2enlo leups bad unl S200) B16 noonaania edd al yt Jpust ont 2 c At J 


ut “ei dansroteng tacdyld ant. ceatat tet ammages teotatta a 

ant nf viata tippeit odw send thie tartusep zevtennoltes 
ovis at sow viel i@tanoged? olear peor saan yd bawol fot 4 - 226 > : 
baton oat (tdkenniie a seu seodd ia lw. elisa bas so08 leak lig o 
| | MmPurisen! Teatetls bas noorze6t i aan 


bexqusoo sanetattih rise inpte s Catt cerestbat 88 ofdah»: | 


fae 7) 
etead ov ihsetowo % say’ oll yo*- equow 9at to oonenst ang ony ow nigone th 


cod wont Jqsveti ih od ah Hoot wie ZIVIou7tT ant nop eats — 
ef odoride t200 SISA IGOT WOT 228  eotesterq 2tit nb oe i 
(aufis het ofW e2odt ya bowel (ot .wesd evMostorg Te PP sce 7 87 7 - 
ofw. szott ud AUB «how ants te tat enoqage tavtegta one von aesta 
brit, «font zatgafont OS otdaT _,2asis fatality eit ah Os 
vietouiten? wovteen so srt sepelted beriyad0 anor squcrtsg | fin une “a 
,2en vigagt Wo. Say any 107 esenonateng, ealataanes ware se 

veiled Yo seyoant to? pcorueerumin geet oe see af 

anti ili dt enoneer mooreEr>’ bee Foshan ts lape i. it) saat 


eH Laatritta ges oy 


Table 26 


Effect of Major Teaching Responsibility on Perceptions 
of Actual and Preferred Data Gathering Practices by Means 


one 


: ' Responsibility F Ratio 
Data Gathering Practice Classroom Clinical Classroom/ 
Clinical 

Performance observation 

ACTUAL 1.40 1.88 1.63 1.06 

PREFERRED 3o00 3.35 3.45 0.20 
Interaction analysis 

ACTUAL 1.40 1262 1363 0.25 

PREFERRED 3.40 peed 3.30 0.13 
Anecdotal records 

ACTUAL 12.30 162 1a/4 0.87 

PREFERRED 3.50 3.06 Shoals O71 
Microteaching 

ACTUAL 1.30 1.33 1.38 0.07 

PREFERRED 3. 30 3.04 3.39 Bede 
Checklists 

ACTUAL 12330 lee 1.74 1.03 

PREFERRED 3220 2.59 ee 1.68 
Rating scales 

ACTUAL 1.30 e57 1.81 1.61 

PREFERRED 2.80 2.41 n4e 0.59 
Inventories 

ACTUAL 1.00 1.16 ij ote: 2.68 

PREFERRED 2.80 Cie Al 2.19 io7 
Ques tionnaires 

ACTUAL 1.20 1.20 Tez 0.00 

PREFERRED CeO 2.06 1.85 3.80* 
Student gain 

ACTUAL 230 eee 2203 Ores 

PREFERRED 3.50 2.69 Liedi3 2220 
Projective tests 

ACTUAL Teh 1.04 1303 0.48 

PREFERRED 1.90 hs ihe liso 4.8] ** 
Teacher tests 

ACTUAL 1.00 Walle Tally 0.61 

PREFERRED 2.90 fo NY 2.07 4, 39** 
Self appraisal 

ACTUAL i 810) hehe: 2.40 0.03 

PREFERRED 4.60 3.96 4.08 2.18 


pal hh rp tt A eet tnt NT 


**Significant at .01 level 
*Significant at .05 level 


99 


- 
2 
—~ 
“tof 


"oe 


ii 


r . PARE ean ABR 


a7 


SOO re 


100 


Table 27 


Effect of Number of Faculty on Perceptions of Actual 
and Preferred Data Gathering Practices by Means 


‘ : Number of Full-Time Faculty F Ratio 

Data Gathering Practice <10 10-19 90-29 30-39 
Performance observation 

ACTUAL e 2°07 35 242i) 1.12 ile 63" *%* 

PREFERRED GuoOn mETGECO 3.69 ELSI WAa2 
Interaction analysis 

ACTUAL Cone oe eae 1.38 1.74 

PREFERRED S217 3. 34 3.12 Bab Tess 
Anecdotal records . 

ACTUAL 1.79 1.54 Lieve 1.29 oat ts 

PREFERRED S200 3024 3.14 2.98 0.69 
Microteaching 

ACTUAL Teo0 ey: 1565 105 4 -23** 

PREFERRED 3.00 Bay. Sal) Sal7 0.93 
Checklists 

ACTUAL 1.79 Loy 1.96 1.39 2530 

PREFERRED 2.80 2.69 2.79 2293 0.41 
Rating scales 

ACTUAL Sey? steerer 2202 1.41 ZOU a 

PREFERRED Deel 2.45 Apes) 2.43 0.43 
Inventories 

ACTUAL ZS W387 1 7 ARAL 2 126 

PREFERRED deel Be eeoe Peal 2.20 O55] 
Ques tionnaires 

ACTUAL 1.39 Wal 1213 Te 1350 

PREFERRED 2.29 1.98 163 2207 3. 73** 
Student gain 

ACTUAL Dal Leto 2.00 1.68 1.99 

PREFERRED 2.90 2.68 2.76 DIG 0.25 
Projective tests 

ACTUAL 1.00 1.05 1.00 1507 0.99 

PREFERRED 1.50 1.41 1.28 1.29 le2zz 
Teacher tests 

ACTUAL 431 Tals 1.13 1.07 132 

PREFERRED Zee ADs 1.88 2ee0 1.65 
Self appraisal 

ACTUAL AStaS: 2.61 Fass S¥e) 2.24 0.85 

PREFERRED o.93 4.07 4.04 4.26 0.90 


*kkS i gnificant at .001 level 
**Significant at .01 level 
*Significant at .05 level 


on perceptions concerning data collection practices. The number of 
full-time faculty had the most effect upon perceptions of any of the 
independent variables. Table 27 indicates that 5 out of a possible 
24 comparisons were statistically significant. The table shows that 
the majority (4 out of 5) of the significant differences were related 
to actual data collection practices. 

Table 27 presents a finding which indicates a significant 
difference in perceptions concerning the actual use of performance 
observation. The smallest schools (those with a faculty of under 
10) were different from the largest schools (those with faculties of 
30-39). Each faculty was also different from the one of the next 
largest size. Schools with 10-19 faculty saw the greatest use being 
made of performance observation. They were followed by schools 
with under 10 teachers, those with 10-19 and, finally, by the largest 
schools (30-39 faculty). 

Table 27 indicates that a significant difference occcured 
among the groups concerning the actual importance which was given to 
anecdotal records. The difference occurred between schools with 20-29 
faculty members and the next smallest (10-19) as well as with the 
largest schools (30-39 faculty members). Schools with 20-29 faculty 
members saw the most use being made of anedotal records. They 
were followed by the smallest schools (under 10), the next largest 
schools (10-19) and finally, the largest schools (30-39). 

Table 27 reveals a difference in perceptions concerning the 
importance which microteaching had as a data collection practice. The 
difference occurred between the largest and second largest schools. 


In terms of the extent to which the practice was utilized, schools 
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with 20-29 faculty saw the most emphasis being given it in present data 
collection practices. They were followed by the smallest schools, 

the next largest and, finally, the largest schools. 

| Table 27 presents a difference concerning the perceptions 

of the importance given to rating scales in actual instructor 
evaluation. The difference occurred between the largest and the 
second largest schools. Those in schools of sizes 10-29 

saw rating scales being used most extensively. They were followed 

by the 10-19 group, the under 10 group and finally, the largest 

school group (30-39 faculty members). 

Table 27 indicates only one significant difference concerning 
preferred practices. The difference occurred between the group 
representing the smallest faculties (under 10) and the third largest 
schools (those with 20-29 faculty members). In order of importance, 
the smallest, the largest, then the intermediate sized schools 


preferred questionnaire useage. 


CRITERIA FOR EVALUATING TEACHING EFFECTIVENESS 


Importance and Consensus of Perceptions 

One of the problems which the study dealt with was to deter- 
mine the extent to which nursing instructors shared common perceptions 
concerning the importance which various criteria did and should have in 
the evaluation of teaching effectiveness. In addition, the study sought 
to determine the degree to which instructors held common perceptions 
concerning actual and preferred emphasis on product, process and presage 
categories of criteria. In order to determine the emphasis which the 


study subjects perceived, it was necessary to ascertain whether or not 
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the criteria utilized in the instrument could be placed in the 

three categories as described by Mitzel. A four, three and two 
factor solution was attempted utilizing preferred criteria for the 
purpose of determining how these items could be categorized. 

| The factor analysis was done using the preferred criteria 
since there was less variance in responses for this situation. When 
the four factor solution was. attempted, one of the factors had an 
eigen value of less than 1. With the three factor solution, 

study criteria which had been predicted as process and presage tended 
to merge into one factor loading. The second factor contained some 
of the items predicted to be product, while the third factor 
consisted of a variety of items. 7 

The two factor solution seemed most suitable. Table 28 
identifies the 29 out of the 30 criteria which loaded significantly 
on one of the two factors. Factor I contained 19 items which appeared 
to reflect a process-presage dimension; Factor II contained 19 items 
which seemedto represent the product categorization. Table 28 shows 
that item 2 (asks thought-provoking questions) did not load 
significantly on either item. 

The 30 criteria selected from the pilot study for inclusion 
in the Nursing Instructor EvaluationInstrument had represented equal 
numbers of process, presage and product criteria as identified by the 
pilot study factor analysis. The merging of the process and presage 
items which is apparent in the factor analysis of preferred criteria 
from the Nursing Instructor Evaluation Instrument may have been due 
to the modification of pilot study items which was done in an attempt 


to make criteria for the Nursing Instructor Evaluation Instrument more 
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Table 28 


Factor Loadings for Preferred Evaluation 
Criteria Emerging from Varimax Rotation 
and a Two-Factor Solution 


(N=184) 

Items Communalities Factor | Factor II 
] Oh shi 0.504 0.321 
2 waa brie) OS315 0.280 
5 Oz256 -0.902 0.506 
4 0.290 0.526 Osa vs 
5 0.279 0.502 0.164 
6 0.232 0.429 0.218 
. 0.307 0.524 0.181 
8 0.211 0.029 0.458 
9 0.431 0.142 0.641 

10 08507, 0.764 02055 
1 0.608 0:°775 0.086 
12 0.499 0.639 0.203 
13 OF6 07 0.786 -0.013 
14 0.553 Ope 2214 
dis) 0.485 0.659 Oe225 
16 0.464 0.668 Omi 
lis 0.268 OF427 0.291 
18 OES, 0.301 0.653 
19 0.580 OeS10 0.696 

20 0.347 0.566 0.164 

21 0.41] 0.558 Os 17 

22 0.574 0.193 0.733 

23 0.623 0.264 0.744 

24 0.491 0.583 0.389 

25 Oeesy, 0.453 0.179 

26 0.348 0.490 0.329 

at 0.462 0.638 0.234 

28 Ges7s Ops il 0.694 

29 0.604 0.293 0.720 

30 0.511 OeZ23i 05677 

Percentage of 

common variance 100.000 79.6 20.4 
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specific and behaviourally-oriented. 

Table 29 presents the importance which various criteria were 
given in the actual and preferred situations by presenting a ranking 
of means for both situations. Table 29 shows that items 25 (follows 
institutional policies), 27 (follows through on commitments), 11] (keeps 
students informed of their progress) and 24 (sets an example for his/her 
students) were ranked as the four most ao ntant criteria in the 
actual situation. Only item number 11 remained in the top four ranking 
for the preferred situation. The top ranking item in the actual 
situation (item 25) received a rank of 21 in the preferred situation, 
indicating that respondents preferred that much less emphasis be given 
to adherence to institutional policies as criterion for assessing 
teaching effectiveness. Item 25 showed the most difference in rank 
(21) between actual and preferred means. Item 27 which ranked second 
in the actual situation received a rank of sixth in the preferred 
situation, while items 1] and 24 were rated first and seventh respectively 
in the preferred: situation. 

Table 29 shows that item 11 moved from an actual rank of 
3 to a preferred rank of 1. Table 29 indicates that item 13 (gives 
his/her students constructive criticism in an appropriate manner) was 
considered the second most important criterion in the preferred situation. 
Both item 13 and item 11 seemed to deal with the teachers’ evaluative 
skills. The table shows that the third most important criterion 
in the preferred situation was item 16 (establishes an environment 
which is conducive to student discussion and expression of feeling). 
The fourth ranked criterion was that of making the students aware of 


the teacher's expectations (item 10). 
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Table 29 


A Comparison of Perceived Importance of 


Actual and Preferred Criteria 
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Criterion 


Mean Score Mean Rank 
Number Category Actual Preferred Actual Preferred 
] P/P SS is 4.20 3 9 
2 - 2.94 4.12 ee 12 
3 P 2.90 Ome 23 25 
4 P/P 3.61 4.40 6 i 
5 P/P Sil by 4.16 16 10 
6 P/P 3.04 4.01 19 15 
7 P/P 2.97 3.93 21 tied 
8 P ona 2.84 W 26 
9 P 2.70 3567 26 23 
10 P/P 3.48 4.52 9 4 
11 P/P Sas 4.62 oe ] 
12 P/P B23 qe 33 14 8 
13 P/P Sie: A255 8 2 
14 PLE 3.47 4.49 10 5 
Ws P/P 3.62 4.39 5 7 
16 ls 3. 36 4.54 1] 3 
17 E7e By ee 3.82 25 20 
18 P 6.0 4.07 18 ine 
19 P Shes) 4.20 ike} 9 
20 P/P 3.34 oro i 24 
21 P/P Bros 4.14 i 1] 
22 P Oace 3.96 15 16 
ra P g.0)] 4.03 20 15 
24 P/P 3.65 4.40 4 7 
25 P/P 3. 86 Bao ] 22 
26 P/P 2.89 3.92 24 18 
(ay P/P Grol 4.44 2 6 
28 P 3.04 4.02 19 14 
29 P 3.14 Sit ny 21 
30 P 2.90 3.84 oes 19 
Rho = .72; significant at .01 level 
P = Product 
P/P = Process-presage 
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Table 29 shows that the Spearman rho was .72. This value 
Suggests that there was a positive correlation between the existing 
and preferred use of the criteria; that is, the criteria perceived 
to play a major role in the existing situation tended to play a major 
role in the preferred situation and, converses criteria perceived 
to play a minor role in the existing situation tended to be seen as 
playing a minor role in the preferred situation. 

Table 29 indicates the categorization of the 30 criteria 
according to the results of the factor analysis. It allows comparisons 
to be made to the importance which the various categories of 
criteria were given by study subjects by presenting a ranking of means 
for the actual and preferred responses. | 

Table 29 shows that the product erutey enn which ranked 
highest in the actual situation (item number 19) received a rank of 12 
out of 30. Item 19 also was ranked as the highest preferred product 
criterion. It received a rank of 10 in this situation. Table 29 
indicates that 7 of the 10 product criteria rank between 10 
and 20 in both the actual and preferred situations. The remaining 3 
criteria ranked between 21 and 30, again for both situations. The high 
rankings which the process-presage criteria received in the actual and 
preferred situations and the low rankings obtained by the product 
criteria indicate that nursing instructors perceived actual and preferred 
criteria which focused on teacher characteristics and teaching behaviors 
to be of prime importance in evaluating their teaching effectiveness. 

Table 30 indicates the extent of consensus in perceptions 
concerning the importance which various criteria were given in the 


actual and preferred situations by presenting the standard deviations for 
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Table 30 


Extent of Common Perceptions Concerning 
Actual and Preferred Criteria 


Criterion Number Standard Deviation 
ACTUAL PREFERRED 
] 1.14 0.79 
2 to 0.88 
3 1,07 1,05 
4 dhesl is: 0.64 
5 1.05 Ons 
6 [eric 0.89 
7 1.20 0.87 
8 Weeo 0.97 
9 1.00 0.91 
10 1.06 0.60 
11 104 0.58 
12 Tel 0.69 
13 Tet hc Cray 
14 ‘lee 5 0.64 
15 1.14 0.69 
16 1.19 0.63 
17 AP 0.87 
18 ea 0. 84 
19 1.11 0.82 
20 Las 0.90 
21 1.04 0.80 
22 ici OE 0.89 
23 Ws lke 0.93 
24 1.06 0.67 
25 1.09 0.98 
26 Thadll 0.88 
fed) 1.06 0.71 
28 1.14 0.88 
29 12.05 O2o% 
30 1.09 1.00 
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each criterion. Table 30 shows that the most agreement concerning 
importance occurred for the use of items 9, 11, 21 and 22, in that 
order. The table also indicates that participants shared the most 
differing perceptions concerning the importance which items 7, 8, 14 
and 16 were given in evaluating teaching effectiveness. 

Table 30 shows that, within the preferred situation, items 
10, 11, 13 and 16 received the most agreement. The table also shows 
that the study respondents shared the least common perceptions 
concerning TonTnOrtence which items 3, 30, 25 and 8 should receive. 
Items 3, 30 and 8 were product criteria while item 25 was concerned 
with the instructor following institutional policies. 

Table 30 shows that the standard deviations for each of 
the preferred criteria were smaller than those for the actual criteria, 
indicating that met rictore were more similar in their perceptions 
of criteria which should be utilized than in their understanding 
of criteria which were being used in existing evaluative practices. 
Differences Between Actual and Preferred Criteria 

The study was also concerned with whether or not any differences 
which occurred between the actual and preferred evaluation criteria were 
of statistical significance. Table 31 shows that a significant 
difference was found between the actual and preferred perceptions of 
evaluation criteria for 29 out of 30 criteria. Table 31 also 
indicates an increase in means between the actual and preferred situations 
for 28 out of 29 significant criteria, indicating that in the 
majority of cases study participants preferred that greater emphasis 
be given to the individual criteria than was presently being done. 


Table 31 shows that the one significant criterion which had a lower mean 


ha 


NG 
=! im. > 
told : 


ales dreresrige aden outs salad aia " dry 
rrlusgo s: 


sed7 at SS bw TS 17 2 saint vaiane hr uase 

foom att darerde ztnagto hve, ses 2atéotont oats pes hae 

bP gk aiagd dite Roinement ent jetnnbiriha Son » a 
_peanev itost*s patroee? enidealsye int oll ola ho 


awet® [we lseur he banveterg ant ataiiw gta zwore Oe afde? - “ae 


. ‘ 


aaa 


‘ot ~ 


acl Ul 


«7 


at in 


~ 
shal 


awoile o2le eiadrd od! .tmeneaqge Peom ot bovisosy 3/ baa Ef, tf 1h 7 


mroitgsovaq sommon t26e7 21 bevels ey uabangenn ybud z srt tet 


—— 


a 


avigvan bT vod? @ bos OS LOE .f amex? dahiw sons end pnbmsgnas 


i 
2 


bawastop gew 2S mast sfdw atest Sauber aay 8 ive 02 .E ett if 
i t 
‘te 
a 
— 


fonohdud tran? pebwatto? aagouvent odd fe 


-29folfod 


is (ows “ot onolpeivah Drebrwae oft Sand gwode OF oldsT 


vik : 


ptvét ho (audod ont sot o@odd Wait) yollame orsw ehsdrts verde 
~~» 
om 


infhoarevobm vot? uf dew bes cility ed bfuote aotiiw nineatas, 10" 7 


dn iduer reg, todd nt tei ihe’ sian ovse aotourten! Tory prttaatbar 


,Mxbioeng avlteolses gotkerne'al ioe pated evew fotiw ateadiis to | 
Siyatins Dore and bps | beaoAl feswot eee} ice t ss 
sou Stiyeits nelaileve benateie ban Fenrse add noowed beriwode eh ; 


eopagaseeth une ton wo sadtodw fittw beqvsonco o2ls gow ybute eaT | 


*peaitiogts: a jens ewona Th oldeT .sonaehrtepre lect enivere YO. 7 

to gwattqaown bevistero bre feutos sit tositod bagd) eew conerstttb ne 
ozte fe etdet .et4siitto OF Yo tuo 8S Wh atid hye i oe 
eantiebate damsterg bab Isso Id Naawed angen nt Staton Me “a 
afd of tertd gotiaotbat witisi ts sano taghe eS Yo. 

alensgna “adasyp Sond berretere atmagtotrvag bute wan ev 

<anod exited ylineaer esw moat evading (eat at 38 


baad “ee * and sd alata apenas 


a 


A Comparison of Perceived Importance of 
Actual and Preferred Criteria by Means 


Table 3] 


Criterion Number Mean Mean T Value 
ACTUAL PREFERRED 
] 323 4.20 -10.43*** 
2 2.94 Aeni2 -13.19*** 
3 2.90 See - 3,59*** 
4 3.61 4.40 - 9.02*** 
5 9 ald, 4.16 -13. 16*** 
6 Bee 4.01 -12.34*** 
7 2.97 3.93 -11.07*** 
8 314 2.84 3.29%** 
9 eT) Shalev -12.4q*** 
10 3.48 4,52 -13.03*** 
1] Sa 4.62 -11.20*** 
12 Gaal) 4,33 -12.19*** 
ih 3.51 4.55 -12.50*** 
14 3.47 4,49 -10.46*** 
15 3.62 4,39 - 9,11*** 
16 Bes6 4.54 -12.86*** 
(Po; 2.83 3.82 -12.06*** 
18 3.10 4.07 -12.15*** 
19 3. 33 4.20 -10.56*** 
20 3.34 3.65 - 3.71|*** 
21 3.08 4.14 - 7.55*** 
(ls Geed 3.96 - 9.47%*** 
23 Seu 4.03 -12.26*** 
24 3.65 4.40 - 9.36*** 
25 3. 86 3.75 Lu20 
26 2.89 3.92 -11.48*** 
on. 3.81 4.44 - 8.95*** 
28 3.04 4.02 -12.44%** 
29 3.14 3.8] - 9.04*** 
30 2.90 3. 84 -11.12*** 


*kkSignificant at .001 level 
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in the preferred situation was item 8 (success of his/her students on 
Canadian nurse registration examinations). The criterion which did 
not show a significant difference between the actual and preferred 


Situations was item 25 (follows institutional policies). 


Effect of Independent Variables on Preferences for the Criteria 
Categories 


The study also considered the effect which the selected 
personal and professional variables had upon the instructors' 
preferences for the use of process, presage or product categories 
of criteria. In order to compare the process-presage category of 
criteria with the independent variables, an average mean for each 
of the sub-groups within the independent variable was calculated. 

This was done by summing the responses of the sub-group items 
identified as process-presage and then dividing by the number of 
responses. The T test was then utilized to determine the effect which 
the variable had upon preferences for process-presage criteria. 

The same procedure was carried out to determine the effect which each 
independent variable had upon preferences for the utilization of 
product criteria. 

Table 32 presents the effect of the dichotomous variables, 
type of present employment, hours of employment and type of program 
on the preferences for the various categories. Table 32 shows that no 
statistically significant differences occurred between the sub-groups 
for each of these variables. The table indicates that whether instructors 
were employed on a sessional or permanent basis; full or Jess than 
full time; or in a baccalaureate or diploma nursing program, they 


possessed similar preferences concerning the use of product and process- 
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Table 32 


Effect of Independent Variables on 
Preferences for Product and 
Process-presage Criteria by Means 
(Using the T Test) 


Process- 
Variable presage 7 Value Product T Value 
Type of 
Employment Sessional 4.2] 3.88 
-0.05 PRM 
Permanent 4.2] 3/3 
Hours of 
Employment 50-74% of full time 4.24 3.94 
0.26 ood, 
75-100% of full time 4.2] 34/3 
Type of 
Program R.N. 4.20 3.74 
-1.30 -0.80 


BaGe 4.33 aE) 


Ms 


presage criteria. 

Table 33 shows the effect which the 6 continuous variables 
had on instructor preferences for specific types of criteria. Table 
33 indicates that 3 out of a possible 12 differences were statistically 
sigiieteant: It shows that level of education, type of program 
and school size had no significant effect upon preferences for the 
use of process, presage and product criteria. 

Table 33 shows that instructors of varying ages differed 
significantly in their preference for the use of product-type criteria. 
The difference occurred between the over 40 age group and the under 
30 and 31-35 age groups. Those over 40 had the strongest preference 
for product criteria. With each successively younger group, the prefer- 
ence for the product criteria decreased. 

Table 33 also indicates that amount of teaching experience 
affected the study participants’ preferences for product criteria. 

The significant difference occurred between those with the least amount 
of teaching experience (less than 3 years) and those with most 
experience (over 8 years). Those instructors with more than 8 years 
had the greatest preference for product criteria. They were followed 
by those with 3 to 4 years experience, those with 5 to 8 years and, 
finally, those with less than 3 years. 

Table 33 shows that those employed for varying lengths of 
time differed significantly in their perceptions of the importance 
which both process-presage and product criteria should have. In each 
case, the difference occurred between those employed the least length 
of time (less than 1 year) and those employed for the longest (5 years 


and over). Those employed for the longest time had the greatest 
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Table 33 


Effect of Independent Variables on 


Preferences for Product and 


Process-presage Criteria by Means 


Variable 
Age < 30 
31-35 
36-40 
40 
Level of R.N. 
Education BESc: 
Masters 
Years of <o3 
Teaching 3-4 
Experience 
5-8 
28 
Years of a 
Present 1-2 
Employment 
3-4 
5+ 
Major Classroom 
Teaching sah 
Responsibility Clinical 
Classroom/ 
Clinical 
Number of <1.) 
Full-time 
Faculty EK 
20-29 
30-39 
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preference for the use of both process-presage and product criteria. 
As the groups decreased in their length of present employment, 
their preference for the use of process-presage and product criteria 


also decreased. 
COMMENTS ON THE STUDY 


Respondents were invited to make comments concerning the 
evaluation of nursing instructors or about the study itself. Thirty 
five percent of the study participants made such comments. The large 
number of comments (107) was taken to be an indication of the nursing 
instructors' concerns about teacher evaluaton. 

The comments were classified under the following headings: 
1. General reaction to the instrument 
2. Types of evaluators 
Evaluation criteria 
Importance of nursing instructor evaluation 


Current nursing instructor evaluation practices 


Oy ot  f& "Fa 


Issues related to nursing instructor evaluation 

Table 34 presents the frequency and percentage of responses 
in each category. The table shows that each category (with the 
exception of number 4) contained an approximately equal number of 
comments, indicating that study participants had chosen to express a 
variety of ideas and concerns. 
A sample of comments from each category is included in the 


discussion following. Comments chosen for inclusion were those which 


{APs 


represented the major concerns within a category and those which presented 


contrasting points of view. The comments included have been reproduced 
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Table 34 


Comments by Category and Frequency 


Frequency Percentage of 
Gar edoyy, of Responses Responses 
1. General reaction to instrument 22 20.6 
2. Types of evaluators 20 1857 
3. Evaluation of criteria 21 19.6 
4. Importance of nursing instructor 
evaluation 7 6.5 
5. Current nursing instructor 
evaluation practices iy 15.9 
6. Issues related to nursing 
instructor evaluation 20 18.7 
107 100% 
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as originally written; although in some cases only excerpts have 
been taken from lengthy comments. 
General Reaction to the Instrument 

Most of the comments in this category dealt with difficulties 
in responding to the sections of the instrument which dealt with 
actual perceptions, concerns related to the rating scale used in the 
instrument and the length of the instrument and comments on the 
clarity, specificity or relevance of specific items. 


Difficult to respond to as there is little input given 
to instructors at our school as to their performance. 


Found actual hard to appraise when not actually being 


done. Would almost have preferred to rate criteria from most 
important to least important. 


I've noted areas where "0" would have best indicated 
the importance of an item. 


Well designed because required thought to answer. Rather 
complex format is explicit in directions. I took 45 minutes 
to do this--too long for teachers not well motivated to answer 
ite 

II C (1) How do you define or describe “enthusiasm"? 
Could be more specific. Perhaps some of your other points would 
lead to or show "“enthusiasm"--I think that in itself is a value 
word. 


17 - "shows interest"--to me that is vague. A more 
definite statement, as the others, could be--makes a contribution 
to the nursing profession. 


24 - example for what? If for professional attitudes and 
actions, yes. If for personal lifestyle and values, no. 


25 - Utilize policies--important up to the point where the 
patient's best interests are not being served, then should utilize 
professional discretion and appropriate lines of authority 
(legality) to attempt to modify the situation. 


Types of Evaluators 


Comments were made concerning each type of evaluator included 


in the instrument. Peer and self evaluation were most frequently 
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mentioned. Comments often related to how a particular type of 
evaluator could be utilized most effectively. 


School director evaluation--valuable if director skilled, 
if available and if done regularly. 


All evaluation from superiors is done on a subjective basis 
rather than using specific assessment tools. Due to the subject- 
ivity involved--the real criteria for evaluation are support of 
administration, maintenance of the status quo, and teaching 
Pee which are not too far removed from the traditional 

ecture. 


Most instructor evaluations are inaccurate because information 
is all second hand or "hear-say". The most effective ones I've 
had given me were from my "immediate" superior who came to observe 
my work for the purpose of evaluating it and helping me improve 
in any way she could. These evaluations were geared to my 
work and the improvement of the nursing program and were not 
personality analyses. 

The most obvious reason they were helpful is that the person 
doing it was secure in her position and knew what she wanted 
from the instructor and could demonstrate to you areas she felt 
you could improve. 


Peer evaluation--should be the best since other instructors 
of same level of students know how we should be functioning. 
Excellent opportunity for sharing--could be done in group sessions. 


Other professional peers should have input to evaluate such 
as head nurse or coordinator of your clinical area. 


I feel that instructor self evaluation is of extreme importance 
and if used consistently with guidance can be the basis for change 
and improvement. One method of achieving satisfactory self 
evaluating is to set specific objectives and goals for yourself 
and, over the specific time period, take a look at these and 
constantly check to see if you are progressing or changing as 
needed. 


Re student evaluations--due to wide and wonderful variety 
of students evaluations, I have some doubts about the validity of 
some. My comments are: 


1. Should be signed by student and open to query as we 
do for student evaluation. 

2. Students should receive some groundwork in evaluation 
technique. 


I feel that students are an important source of evaluation 
for the instructor. But--the areas students are evaluating should 
be appropriate to their level of competence and insight. (e.g. 
hard to evaluate certain aspects of a course such as content 
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when new to the material. Good to evaluate the rate of material 
covered. 


Is need to look at total picture, for this need input from 
Students as well as peers, and nursing service staff working with 
instructors. 


Who evaluates is important--does the person know about 
teaching; is she a respected and good teacher herself? What 
of personal bias? Possibly evaluation by more than one person 
would be best and help achieve more objectivity. 


Evaluation Criteria 

The majority of comments in this category were made concerning 
the use of student gain (product) criteria in measuring teaching 
effectiveness. A variety of concerns were presented. Other comments 
suggested additional criteria which might be used to evaluate teaching 
and how specific criteria might best be utilized. 


Evaluation is, of course, presently confronting many 
professional groups--teachers in particular. The public cries 
for "accountability"--yet has limited understanding of the task 
of teaching. Your objective about measuring the teacher's 
effectiveness by the marks his/her students achieve is frightening 
and does not recognize the individual student's responsibility 
and individual. differences. 

Teaching and learning are not synonymous. They are both 
merely parts of a much larger picture. 


Although student performance seems to be one way to evaluate » 
instructors, the success or failure of the student may be due to 
course structure rather than teaching ability, or to individual 
student differences. 


Great emphasis is placed on the progress of students--is this 
realistic, if nursing instructors have no input into student 
selection, and therefore perhaps may have several marginal students 
in a class who either fail out of the program or fail their R.N.'s. 


I think instructors should also be evaluated re their use of 
time. I feel that many hours of time are "wasted" even though 
instructors are always "busy". 


I found Part C on page 5 difficult to answer since I feel 
that the importance that each criterion is given depends on whom 
is doing the evaluation e.g. student or administrator. 


I felt that some of the criteria and their level of importance 
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would depend on: 


1. the ratio of students/instructor 
2. the level of students 


I feel there is a real need to develop criteria for evaluating 
instructors. We spend a lot of time improving our evaluative 
techniques with students, but much less effort goes into evaluating 
an instructor. 


Importance of Nursing Instructor Evaluation . 


Comments in this category indicated that nursing instructor 
evaluation was seen as a pertinent and relevant topic for investigation. 


Nursing instructor evaluation is very important aspect 
especially as so many programs are now using sessionals who have 
very little contact with full time faculty. Instructors need 
assessment and assistance in developing their skills--I hope 
this study will be distributed to all Schools of Nursing and 
that the results will provide useful recommendations . 


I'm glad to see you researching this very pertinent issue 
in Nursing Education and hope that your data will shed some light 
On meaningful criteria etc. for nurse educator evaluations. 


Current Nursing Instructor Evaluation Practices 


The majority of comments in this category were made concerning 
the adequacy of present evaluation practices. In some cases uncertainty 
was expressed concerning what procedures were utilized. 


The institution wherein I am currently employed places the 
greatest emphasis on instructor evaluation which I have been 
exposed to in 9 years of teaching. Because of my experience with 
limited evaluations of my performance prior to 1975, I am very 
pleased to see work being done on this topic as there would seem to 
be a need for it. 


I have been evaluated once in 4 years, by the Director of 
Nursing Education, using a rating scale designed by Personnel 
to evaluate any employee. I found this an exercise in futility: 
the rating scale was not specific for my job description, the 
evaluator based her comments from my written evaluation of my 
students and their comments about me, and from minimal verbal 
feedback from one of my peers. At best, this evaluation reflected 
the Director's perception of me from second-hand sources. 
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My experience concerning nursing instructor evaluation has 
been rather limited in that formal evaluations were seldom given, 
only on termination, and I felt these were very subjective at best. 


I have not been evaluated in this school of nursing for the 
last three years therefore, I do not know the criteria which are 
used. 


Instructor evaluation (in my experience) is usually carried 
out on thebasis of very little contact in the work area between the 
instructor and evaluator. Evaluations then tend to be entirely 
positive and complimentary; the instructor must then recognize 
her need to continue self-evaluation on an on-going basis in order 
to assess effectiveness. 


I would value more feedback on my performance from a trained 
evaluator who has no vested interest in my career or control over 
my position. (I appreciate that my students are in just the 
particular bind I wish to avoid . . . . their promotion depends 
greatly on my evaluation. ) 


The A.A.R.N. collects a substantial fee . .. . could they 
use some of the funds to provide such a service .... an 
on-call observer . . . . how's that for a dream: 


Issues Related to Nursing Instructor Evaluation 


A wide variety of comments were made in this category. 
They dealt with such concerns as the purposes of evaluation, objectivity, 
the time required to evaluate effectively and weighting of evaluative 
input. 


The college where I work is heavily into evaluation, not only 
of nursing instructors but all staff. The difficulty we are 
encountering relates to the utilization of evaluation procedures 
for assessment as well as development of staff. As evaluation 
procedures are used for assessment primarily, the value of evaluation 
for development purposes has been neglected. 


How should evaluations be used--for promotion, merit 
increment, dismissal, for progress identification? 


We need objective evaluation which prevents rewarding for 
criteria such as social popularity and extraversion. 


How do you provide time for precise evaluation? 


Nursing instructor evaluation should be done on a regular 
basis in some form other than just a self evaluation. I believe 
it is important for the nursing instructor to have constructive 
feedback in order to grow in his/her job. 
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A thorough knowledge of teaching-learning theory is of 
value when one embarks on evaluations. As evaluation means to 
e-value-ate, one must be cognizant of his/her own values before 
assessing the attributes of a teacher. 

Will the method of evaluation be constructive rather than 
destructive being that the evaluation could be very threatening 
especially to a beginning instructor who is still trying to 
adjust to a situation. 

I think the weighting of data on instructors’ performance 
should be proportional to the length of time the supplier has 
interaction with the instructor, i.e. someone who sees her 
for two hours a week at a meeting should have less impact than 
another person who interacts with her for twelve hours a week, 
in relation to the same criteria of performance. 

Often feel that senior instructors are not evaluated 
realistically--it is accepted that they are performing adequately 
and they are given little input for personal growth. 


Areas in which one is being evaluated should be clearly 
outlined to nursing instructors at the start of their employment. 


The overall picture should be evaluated--by a variety of 
tools and from a variety of sources. 


SUMMARY 


In this chapter the data gathered from study respondents were 
analyzed and discussed. Frequency and percentage distributions were 
presented and discussed in order to describe the personal and professional 
characteristics of the study respondents. Correlations between the nine 
independent variables were shown and discussed. 

A presentation of means, mean ranks and standard deviations 
was used for each of the sections which dealt with evaluators, data 
gathering practices and criteria in order to discuss the extent to which 
respondents shared common perceptions concerning these aspects of nursing 
instructor evaluation. Spearman rho calculations from ranks were used to 


indicate whether any overall differences existed between the actual 
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and preferred situations for evaluators, data gathering practices 
and criteria. 

The results of T tests were used to discuss whether differences 
between actual and preferred evaluators, data gathering practices and 
_criteria were of statistical significance. 

The results of a factor analysis were used to present the 
most suitable categorization of the study criteria according to the 
classification system proposed by Mitzel. 

The results of T tests and F tests were presented and discussed 
in order to show the effect which the nine independent variables had 
upon perceptions concerning actual and preferred evaluators, data gathering 
practices and concerning preferences for the use of product, presage 


or process criteria. 
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CHAPTER V 
SUMMARY, CONCLUSIONS AND IMPLICATIONS 


In this chapter the problem, procedure and results of the 
study will be summarized. Conclusions will be stated and some implications 


for practice and for further research will be discussed. 
SUMMARY 


The nursing profession seems to be increasingly concerned 
with evaluation as part of the accountability issue; however, the 
literature examined by the researcher indicated that relatively little 
attention has been paid to the topic of nursing instructor evaluation 
in Canada. Student evaluative input has been discussed to the greatest 
extent in the existing literature. The use of other types of evaluative 
personnel, purposes of evaluation and methods of data collection seemed 
relatively unexplored. Criteria for evaluating nursing instructor 
effectiveness have received considerable attention although no evidence 
existed conceming the use of the categorization of criteria as proposed 
by Mitzel (1960). 

This study has sought to provide practicing administrators 
and teachers in nursing education with up-to-date information concerning 
the perceptions of Alberta nursing instructors in terms of actual 
and preferred evaluators, data gathering practices and criteria for 
evaluating their teaching effectiveness. 

A pilot study was conducted with British Columbia and Alberta 


nurses who were teaching or had taught in diploma and baccalaureate 
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nursing programs within the last three years. 

A Nursing Instructor Evaluation Instrument was used to 
measure nursing instructor perceptions. The instrument consisted of 
three sections. Section I requested personal and professional 
_information on nine variables. Section II consisted of seven types 
of evaluation personnel, twelve data gathering practices and thirty 
evaluation criteria. The criteria included had been modified from 
pilot study criteria which had been identified as product, process 
or presage by a factor analysis. Instructors were asked to indicate the 
importance of the evaluators, data gathering practices and criteria 
in actual and preferred evaluation practices. In Section III 
nursing instructors were invited to comment on evaluation and/or 
the study. 

Alberta nursing instructors who were teaching at least half 
time in diploma or basic baccalaureate nursing programs comprised the 
sample for the study. The Nursing Instructor Evaluation Instrument 
was used to collect all data and, in order to guarantee anonymity, 
no identifying marks appeared on any of the questionnaires or on any 
of the envelopes in which the questionnaires were returned. Each study 
participant was offered a summary of the findings of the study. 

Frequency and percentage distributions were used to present 
a preliminary analysis on the personal and professional data collected 
from the 184 nursing instructors. Correlational procedures were 
utilized to determine interrelationships between the 9 personal and 
professional variables. 

Ranking of means for both the actual and preferred situations 


was utilized to determine the extent to which respondents shared common 
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perceptions concerning the importance of actual and preferred 
evaluators, data gathering practices and criteria. 

A Spearman rho calculation was used to determine the extent 
of similarities or differences between the ranking for actual and 
preferred evaluators, data gathering practices and criteria. 

Standard deviations for actual and preferred evaluetorss 
data gathering practices and criteria were calculated in order to 
show the extent to which study respondents shared common perceptions 
concerning these aspects of nursing instructor evaluation. 

T tests were utilized to determine if any of the differences 
between the actual and preferred situations for evaluators, data 
gathering practices or criteria were of statistical significance. 

A factor analysis was performed in order to determine if 
the study criteria could be classified according to the product, process 
and presage categorizations. Four, three and two factor solutions 
were attempted on the preferred criteria. The ranking of each 
criterion by category for both the actual and preferred situations was 
determined. 

T tests and F tests were utilized to determine the effect of 
the 9 independent variables on the perceptions of the respondents concerning 
actual and preferred evaluators and data gathering practices and concerning 
the preferred use of product, presage and process criteria. 

Representative nursing instructor comments on the study were 
classified under the following headings: Types of Evaluators, Evaluation 
Criteria, Importance of Nursing Instructor Evaluation, Current Nursing 
Instructor Evaluation Practices , and Issues Related to Nursing Instructor 


Evaluation. 
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The various analyses of the data revealed the following: 

1. The majority of the instructors in the study were 
between 26 and 35 years of age, prepared at the baccalaureate level and 
teaching in diploma nursing programs. In addition, most had instructional 
responsibilities in both the classroom and clinical areas and had been 
teaching nursing for less than 5 years. 

2. Thirty-two out of 36 significant PAE EAI were found 
between the independent variables. The majority of the significant 
relationships were positive in nature. 

3. Senior administrators and immediate supervisors were 
considered to be the most important evaluators in present evaluative 
practices while the study participants preferred that the teacher be the 
primary evaluator. Immediate supervisors retained the rank of second 
most important evaluator in the preferred Situation. Respondents perceived 
that self appraisal, student gain, checklists, and rating scales were 
presently the most important data gathering practices, while the 4 top 
ranking practices in the preferred situation were self appraisal, performance 
observation, microteaching and interaction analysis, in that order. 

The ranking of means for the criteria section of the questionnaire indicated 
that only 1 of the top 4 ranked criteria for the actual situation 
(item 11) remained in that ranking for the preferred situation. 

4. Nursing instructors shared more common perceptions concerning 
who should evaluate and concerning the criteria which should be utilized 
than concerning the evaluators and criteria utilized in present evaluative 
practices. All differences between actual and preferred evaluators were 
of statistical significance except for that difference involving the 


senior administrator(s). All 12 of the differences between actual and 
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preferred data gathering practices were of statistical significance, 

as were 29 of the differences between actual and preferred criteria. 

The mean for the preferred situation was higher than the mean for the 
actual situation in all cases except those involving the senior 
-administrator and the criterion which dealt with assessing the teacher 
in terms of the success of his or her students on the registration 
examinations. The higher means for the preferred situation indicated 
that respondents preferred evaluators, data gathering practices and 
criteria to be utitized significantly more than was presently being done 
in nursing instructor evaluation. 

5. Twenty-nine out of the 30 criteria loaded significantly 
on one of two factors. Factor I seemed to represent a process-presage 
categorization while Factor II appeared to represent those items with 
a product orientation. The process-presage criteria ranked highest in 
both the actual and preferred situations; the highest ranked product 
criterion received a rank of 10 in the actual situation. 

6. The independent variables were found to have an effect on 
43 out of a possible 342 perceptions concerning actual and preferred 
evaluators, data gathering practices and criteria. 

Age had significant effect upon the perceptions concerning the 
following: (1) the actual role of the immediate supervisor (2) the use 
of performance observation (3) the preferred use of anecdotal records 
(4) the preferred use of student gain as a method of data collection 
and (5) the preferred use of the product criteria as identified by the 
factor analysis. 

Level of education had a significant effect concerning types 


of evaluators only. The two significant findings dealt with 
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(1) preference for the use of peer instructors and (2) preference for 
student evaluative input. 

Amount of teaching experience had a significant effect upon 
perceptions concerning the following 5 variables: (1) actual 
_ importance of immediate supervisor (2) actual use of performance 
observation (3) actual use of student gain for data collection 
(4) preferred use of student gain for data collection and (5) preferred 
use of the product criteria as identified through factor analysis. 

The length of time the instructors were employed had a 
Significant effect upon perceptions concerning the following 5 variables: 
(1) actual importance of self evaluation (2) preferred importance of self 
evaluation (3) preferences for the use of microteaching (4) preferences 
for the use of the process-presage category of criteria and 
(5) preferences for the use of the product category of criteria. 

Differences between instructor perceptions for those employed 
On a sessional and a permanent basis occurred concerning the following 
4 variables: (1) preference for student evaluative input (2) actual use 
of performance observation (3) preferred use of microteaching and 
(4) preferred use of projective tests. Type of employment had no effect 
on preferences for the use of process-presage or product criteria. 

Differences in perceptions between those employed 50-74 percent 
of full-time and those employed 75 percent of full-time to full-time 
occurred only in the area of the following 3 data gathering practices: 
(1) actual use of rating scales (2) preferred use of projective tests 
and (3) preferred use of self appraisal. 

Very few differences in perception occurred between 


instructors teaching in classroom, clinical and combined classroom and 
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clinical settings. The significant differences, all of which dealt 
with preferred data gathering practices, involved the use of 
(1) questionnaires (2) projective tests and (3) teaching tests. 

Instructors teaching in baccalaureate and diploma nursing 
programs seemed in agreement concerning data collection practices and 
criteria for evaluating teaching effectiveness. Three significant 
differences in perceptions occurred related to the following evaluators 
(1) preferred use of peer evaluators (2) actual use of students and 
(3) preferred use of students. 

School size (as indicated by the number of full-time faculty) 
influenced the greatest number of perceptions concerning teacher 
evaluation. Perceptions concerning the following 13 variables were 
affected (1) actual use of senior administrators (2) actual use of 
immediate supervisors (3) actual use of peers (4) preferred use of 


peers (5) actual use of self (6) actual use of students (7) preferred 


use of students (8) actual use of performance observation (9) actual use 


of rating scales (10) actual use of microteaching (11) preferred use 


130 


of questionnaires (12) actual use of anecdotal records and (13) preferred 


use of anecdotal records. School size had no significant effect on 
perceptions concerning the process-presage and product categories of 


criteria. 
CONCLUSIONS 


The following conclusions are based on the findings of the 
study: 
1. Nursing instructors perceive senior administrators and 


immediate supervisors to be the most important evaluators in present 
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nursing instructor evaluation practices. They prefer that more emphasis 
be given to all types of evaluators except the senior administrator and 
are highly in agreement that the instructor herself and the immediate 
Supervisor (in that order) should be the most important evaluators. 

2. Nursing instructors prepared at the Master's level, those 
instructors teaching in baccalaureate nursing programs and those teaching 
in schools with 10-19 faculty have a stronger preference for peer input 
than do those teachers prepared at the baccalaureate or diploma level, 
those instructors teaching in diploma programs and those instructors 
teaching in schools of fewer or more than 10-19 faculty members. 

She Nursing instructors do not see or prefer students to be 
a major source of evaluative input. However, those instructors working 
in baccalaureate nursing programs and those instructors prepared at the 
Master's level see and prefer more student involvement than do those 
teaching in a diploma program or those prepared at the baccalaureate or 
diploma level. In addition, those teachers employed on a sessional 
basis have a stronger preference for student input than do those employed 
on a permanent basis. 

4, Nursing instructors prefer that a broader range of data 
gathering practices be utilized in assessing their teaching effectiveness. 

5. Nursing instructors perceive self appraisal, measuring 
student gain, checklists and rating scales to be the major data gathering 
practices used in present instructor evaluation but are highly in 
agreement that self appraisal and techniques which require direct 
observation of the teacher should be utilized. 

6. Nursing instructor preferences for measuring student gain 


as a method of gathering data concerning teaching effectiveness increase 
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directly with teaching experience. 

7. Nursing instructors prefer that a broader range of criteria 
be utilized in assessing their teaching effectiveness. 

8. Nursing instructors prefer that criteria of a process- 
- presage nature receive the greatest emphasis in nursing instructor 
evaluation. Nursing instructors perceive that criteria such as adherence 
to institutional policies and following through on commitments are of 
major importance in the present assessment of nursing instructor 
effectiveness. They prefer that criteria which measure a teacher's 
evaluative and communication skills receive the most emphasis. 

9. Nursing instructor preference for the use of specific 
product criteria increases with age but is not affected by the other 


independent variables. 
IMPLICATIONS 


Although the data provided by this investigation resulted in 
some significant findings, it is recognized that much more empirical work 
is needed in the area of Alberta nursing instructor evaluation. 
Consequently, suggestions concerning implications for practice must be 


expressed cautiously. 


Implications for Practice 


Findings for this study which appear to have relevance for 
nursing instructors and administrators are contained in the section of 
this chapter which deals with conclusions. The findings indicate a 
preference on the part of Alberta nursing educators to broaden the 
range of evaluators, practices and criteria utilized to assess their 


teaching effectiveness. In most instances, the educators prefer that a 
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different emphasis concerning these various aspects of instructor 
evaluation be used. 

The findings indicate that nursing educators are concerned about 
the importance which the senior administrator has in present evaluative 
practices. With an increasing focus on accountability predicted for the 
1970's (Miller, 1974), the senior administrator might do well to 
consider how he or she could best facilitate an effective evaluative 
program. The administrator will continue to be responsible for decisions 
concerning the retention and promotion of faculty and yet the findings of 
this study indicate that nurse educators would prefer that immediate 
Supervisors and instructors themselves play the major roles in evaluating 
teaching effectiveness. One appropriate function for the administrator 
might be to ensure that adequate resources are available for the 
development of evaluative skills in instructors and their supervisors. 

The data gathering practices which were rated as important by 
the study respondents may be worth consideration by those developing or 
revising a nursing instructor evaluation system since it seems that a 
high degree of consensus existed concerning actual and preferred data 
gathering practices. 

The rating of self appraisal as the most important actual and 
_ preferred data gathering practice reinforces the finding that nursing 
instructors see themselves as appropriately involved in both gathering 
data and providing input concerning their teaching effectiveness. The 
preferences for performance observation, interaction analysis and 
microteaching indicate that nursing instructors prefer to have direct 
observation of their performance form the basis for assessment of their 


teaching effectiveness. This preference for direct performance 
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observation might lead to a consideration of who might best observe 
teaching behavior. Administrators may be restricted in the time which 
they have available for regular performance observation while immediate 
supervisors, peers and students may be able to accomplish this process 
more readily. An appropriate role for the administrator might be to 
ensure that adequate time and personnel resources are available for 
regular performance observation to be carried out. 

The relatively low ranking (4 out of 6) which students received 
as evaluators requires further exploration by erinaaetrators and faculty 
members. It may reflect a discontent in those schools where students 
are the major source of evaluative input or a perception that students 
are more skilled at evaluating for some purposes than others. For 
example, the findings concerning students may have been different if the 
questionnaire had considered evaluation for the purpose of instructional 
improvement only. In establishing or revising an evaluation system, 
administrators and their faculties would need to determine appropriate 
roles for students to play in evaluating teaching effectiveness. 

The difference in perceptions between instructors in diploma 
and baccalaureate nursing programs concerning the actual and preferred 
use of student input as well as the preference for peer input may have 
implications for developing or revising instructor evaluation systems. 
The difference may be due to an increased exposure on the part of those 
teaching in the baccalaureate programs to information on the use of peer 
and student input. Administrators may do well to ascertain faculty 
perceptions concerning the formal use of peers in providing evaluative 
input. In addition, as more literature on peer review in nursing 


education becomes available, administrators may want to ensure that the 
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information is given consideration by their instructors. The study 
Findings concerning the importance which peers should play in providing 
evaluative input might have been different if the questionnaire had 
provided an explanation of peer review as a formalized procedure 
(Stone, 1977). 

The findings concerning school size and its effect upon 
data collection practices may reflect a difference in what schools in 
the province are actually doing for nursing instructor evaluation, rather 
than a difference due to size. The comments received from the study 
participants indicated that the schools do vary in how much they are 
presently involved in nursing instructor evaluation. 

The findings indicate that age and amount of teaching experience 
have some effect on perceptions concerning actual and preferred data 
collection practices. For example, one finding indicates that the 
preference for the use of student gain increases directly with the 
amount of teaching experience. Perhaps as a teacher gains experience 
she becomes more confident in making a self assessment by judging the 
performance of her students or in having others assess her performance 
in terms of the progress of her students. Administrators and faculty may 
need to consider individualizing the criteria and methods used to assess 
teaching effectiveness as the perceptions of a nursing instructor 
change during her teaching career. 

The criteria provided in the study may be of some use to 
administrators and faculty in designing or revising instructor evaluation 
instruments. Those criteria which were ranked as the two most important 
in the actual situation (follows institutional policies and follows 


through on commitments) appear to be the kind of criteria which could 
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be assessed without: direct observation of the teacher's performance. 

The low ranking of these criteria in the preferred situation and the 
content of the items which did receive top ranking indicate that 
teachers see items which deal more directly with instructor-student 
‘interaction as being of most importance. This finding would seem 
consistent with the preferences of teachers for the use of various 
performance observation techniques and for evaluative input from their 
immediate supervisors. The findings may have been different if the 
criteria had been indicated as being specific to a particular situation, 
although classroom and clinical instructors and those who shared equal 
classroom and clinical responsibilities seemed to have similar preferences 
throughout the study. 

The age of the study participant had an effect upon the 
specific preference for those 10 criteria identified as product by the 
factor analysis. This finding, in addition to the rank received by the 
product criteria, would suggest that these specific student gain 
criteria might not be effective for use in Svaluating all nursing 
instructors. The comments of Brady participants indicate similar 
concerns related to the use of product criteria as were raised in the 
literature by such authors as McNeil and Popham (1973). 

The findings indicate that a relatively narrow range of 
evaluators, data gathering practices and criteria are utilized in 
assessing nursing instructor effectiveness. A more indepth exploration 
of the options available for effective teacher assessment may be 
appropriate for programs which prepare nursing instructors and for 


inservice development activities. 
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Implications for Research 


Because this study was the first of its kind known to have 
been conducted in Alberta, more empirical research might be carried out 
in an attempt to either support or refute the conclusions of this study. 
While it appears that a great deal remains to be learned about all the 
variables and relationships with which this study is concerned, further 
research regarding nursing instructor evaluation might be more 
productive if the following questions and tasks were considered: 

1]. Further research should be undertaken concerning the 
role which the senior administrator could play in nursing instructor 
evaluation. 

2. Further investigation concerning the role which the immedi- 
ate supervisor does and should play should be undertaken. The results of thi 
Study indicate that nursing instructors see that this individual is and 
should continue to play a major role in providing evaluative input. 

$e Further research into the role of peer evaluation should 
be undertaken. For example, a determination of factors which influence 
attitudes concerning the suitability of peer evaluation could be carried 
out. 

4, Further research should be conducted as to how instructors 
themselves can best play a major role in collecting data and providing 
input to evaluate their teaching effectiveness.. The study participants 
indicated a strong preference for self appraisal practices. 

5. Further investigation into the role which other professionals 
should take in providing evaluative input could be undertaken. The 
study indicated that by far the majority of participants had at least 


some responsibilities in the clinical area and therefore regularly came 
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in contact with charge nurses, staff nurses and other members of the 
health care team. 

6. Further investigation into the role which students should 
play in nursing instructor evaluation should be undertaken. For example, 
_ the purposes for which their evaluative input could be used and the 
weighting given to such input could be studied. 

ie Further research into the use of techniques for the direct 
observation of nursing instructor performance in the classroom, 
laboratory and patient care settings should be undertaken. The results 
of this study indicated that teachers instructing in various settings 
seemed to share a Similar perception that a broader range of data 
collecting practices should be utilized. They also indicated agreement 
as to which ones they would prefer to be of major importance. — 

8. The issue of whether classroom and clinical instructors 
require separate evaluation criteria requires further investigation. The 
results of this study indicate that the majority of Alberta nursing 
instructors teach students in both settings. Having teaching responsibil- 
ities in both areas could influence an instructor's perception of what 


criteria would be considered appropriate. 
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FACULTY OF EDUCATION 
DEPARTMENT OF EDUCATIONAL 
ADMINISTRATION 


THE UNIVERSITY OF ALBERTA 
EDMONTON, CANADA 
T6G 2G5 


March 7, 1977 


Dear Nurse Educator: 


Evaluation of nursing instructors may be a topic of concern 

to you. The purpose of the attached questionnaire is to 

obtain input concerning your perceptions of the process of 
evaluating nursing instructors as it is now and how you 

think it should be. The questionnaire is being circulated 

to Alberta nursing instructors who have teaching responsibilities 
at the undergraduate level. The data received will be analyzed 
and a thesis prepared. The thesis will be available from the 
A.A.R.N. and University of Alberta Education Libraries once 

the study is completed. 


I would ask your assistance in completing the questionnaire 

and returning it in the stamped, self-addressed envelope which 

you will find enclosed. The questionnaire should take 20-30 

minutes of your time to complete. I hope the results of the 

study will be of some value to you and I am willing to send 

you a summary of my findings as a small thank you for completing 

the questionnaire. Enclosed you will find a stamped, self-addressed 
postcard to complete if you wish to obtain that summary. 


Please mail the postcard and questionnaire separately in order 
to ensure the anonymity of your response. 


I am looking forward to receiving your input. Thank you for 
your assistance. 


Sincerely yours, 


Lee Cadman, R.N., B.Sc. 
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EVALUATION OF NURSING INSTRUCTORS 


Personal & Professional Data 


In the box provided, please write the number 
which represents the most accurate answer to 
each section. 


(a) Age to your nearest birthday - 
1. :21).to 25 4. 36 to 40 6. 46,,.to,50 
2. 26 to 30) «5. 41 to 45 7. over 51 
Som ocOr os) 


(b) Highest level of education - 


1. R.N. diploma 
2. R.N. plus post basic diploma in nursing 
3. R.N. plus other, Please specify 
4, Baccalaureate degree 
5. Master's degree. Please specify 
6. Baccalaureate plus other. 
Please speci fy 
he 


Other. Please specify 


[| 


[| 


(c) Total amount of teaching experience in nursing 


education - 

1. Less than 1 year 4. 5 to 6 years 
2. 1 to 2 years 5. 7 to 8 years 
3. 3 to 4 years 6. over 8 years 


o_ 
Qa 
~— 


Length of present employment - 


4, 5 to 6 years 
5. 7 to 8 years 
6. over 8 years 


1. Less than 1 year 
2. 1 to 2 years 
3. 3 to 4 years 


— 


(e) Type of present employment - 


1. sessional 2. permanent 


(f 


— 


Hours employed at present - 


1. 50 - 74 % of a full time position 
2. 75 - 100% of a full time position 


(g) Areas of major teaching responsibility - 


1. Classroom instruction 

2. Clinical instruction 

3. Approximately equal classroom and 
clinical responsibilities 

4. Other - please specify 
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For Keypunch 
(h) Type of program in which you presently Use Only 


teach - 

| | C 
1. Prepares diploma nurses 12 
2. Prepares Baccalaureate nurses 


(i) Approximate number of full time faculty in 


present program a 


1. Under 10 3. 20 to 29 5. over 39 
2. 10 - 19 4. 30 to 39 


is 


Please use the following response key throughout the remainder 
of the questionnaire. For your convenience, the key is 
reproduced at the bottom of each subsequent page. 


Response Key 
] 2 3 4 5 


very limited some moderate great very great 
importance importance importance importance importance 


II. Nursing Instructor Evaluation 


A. The following list indicates types of individuals who 
may be involved in evaluating nursing instructors. 
Using the key provided, please circle the response 
which best indicates your perception of the importance 
which each type of individual has (ACTUAL) and the 
importance that type of individual should have 
(PREFERRED) in nursing instructor evaluation. 


Example Item 


School librarian (Actual) @AT 1 @ 3 2°5 
(Preferred) P 


(Using the key provided, the numbers circled indicate 

that the respondent perceives school librarians as having 
some importance in evaluating nursing instructors; however, 
he/she perceives that they should have great importance in 
evaluating nursing instructors. ) 


Types of Individuals Importance 


1. Senior administrator(s) of Ata higtc ies 
fe 


4 5 
nursing programs EY 4 5 15 


Ti 


Types of Individuals 


. Most immediate supervisors 


. Peer instructors 


. Instructors themselves 


Students 


. Others. Please specify 


type of individual and 
importance 


3 
For Keypunch 
Use Only 


Importance 
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B. The following represents a range of practices which 
may be utilized in gathering information for evalu- 


ating nursing instructors. 


Please circle the numbers 


which indicate your perception of the importance 
which each practice is given (ACTUAL) and should be 
given (PREFERRED) in evaluating nursing instructors. 


hee 


he 


Response Key 
] 


very limited 
importance 


ACTUAL P = PREFERRED 


Description of Practices 


Performance observation - 
trained observers watch 
instructors in the work 


setting and record what they 


do. 


. Interaction analysis - 
analysis of the number and 


kinds of verbal inter- 
tions between instructors 
and students. 


. Anecdotal Records - a ; 
series of notes concerning 


the behaviours of an 


instructor. 
2 3 
some moderate 


importance importance 


Importance 


Ay} 2 
Rar ti 2 
Are? 2 
Pe} 2 
Alt1s 2 
Poe lie 
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great 
importance 
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very great 
importance 
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11. 


Description of Practices 


. Microteaching - analysis of specific 


teaching skills by assessing short 
periods of teaching - most 
frequently videotaping is used. 


. Checklists - using a list of 


traits, skills or behaviours to 
evaluate teachers. 


. Rating scales - comparing the traits, 


skills or behaviours of the teachers 
being evaluated with those of other 
teachers, with norms, or with other 
criteria. 


. Inventories - using a test or check 


list to determine the instructor's 
ability, achievement, aptitude, 
interest, or likes, generally in 

a limited area. 


. Questionnaires - circulating a list 


of planned, written questions 
related to a particular topic - com- 
monly used to measure attitudes 

and opinions. 


. Measuring student gain - appraising 


an instructor by assessing the 
amount of progress made by his/her 
students. 


. Projective Tests - giving an 


instructor some ambiguous or unstruc- 
ured stimulus (such as an incomplete 
sentence or a picture) and asking 
him/her to complete his/her own 
response. 


Teacher Tests - standardized tests 
to gather information about specific 
teacher abilities. 


Response Key 
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2 3 4 


very limited some moderate great 


importance 


importance importance importance 
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5 
For Keypunch 
Description of Practices Importance Use Only 
C 
12. Self appraisal - methods, tech- Aa lise) 2oneaae 48 
niques, material and tools used Pal S28 32.40o 49 
by instructors to gain evaluative 
data about their growth and 
development as instructors. 

C. The following criteria may be used in evaluating nursing 
instructors. Included are selected items dealing with 
characteristics of the instructor, the actual teaching- 
learning process and the concept of evaluating the 
instructor by assessing the performance of his/her 
students. 

Please circle the number which indicates your perception 
of the importance which each criterion has (ACTUAL) and 
should have (PREFERRED) in the evaluation of nursing 
instructors. 
A = ACTUAL P = PREFERRED 
Criteria Importance 
1. Demonstrates enthusiasm for A ol 28 Se 45.5 50 
teaching. PP 29932 49.5 5] 
2. Asks thought-provoking questions. A 1829 32 425 52 
P 1% 27-37 435 53 
3. Success of his/her students in A l@22e032 428 5 54 
meeting course requirements. Pe 3 45 55 
4. Evaluates students based on Avail te moo. Sano 56 
course objectives. Pie 2) 32. 43-5 57 
5. Is well informed on technical Arhlgeemose 4-20 58 
and professional advances. eS ee Tee Bea 59 
6. Interprets abstract ideas and A FP 24-32 43 5 60 
theories clearly. P| emo a4 61 
7. Shares own thinking with Ai 1Pelece See 62 
students. Pav bes Os cas 80 63 
8. Success of his/her students on Al23 4 5 64 
Canadian nurse registration id RY yl hed 65 
examination. 
Response Key 
1 (od 3 4 5 
very limited some moderate great very great 
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Response Key 
] 


very limited 
importance 


Criteria 


Progress of his/her students 
in utilizing communication 
skills. 


Makes students aware of his/her 
expectations for them. 


Keeps students informed of 
their progress. 


Takes advantage of new or 
unexpected situations in teaching 
his/her students. 


Gives his/her students construc- 
tive criticism in an appropriate 
manner. 


Respects individuality of 
students. 


Makes himself/herself available 
to students when needed. 


Establishes an environment which 
is conducive to student discussion 
and expression of feeling. 


Shows interest in making a 
contribution to the profession 
of nursing. 


Progress of his/her students in 
utilizing problem solving skills. 


Progress of his/her students in 


relating theory to nursing 
practice. 


2 3 4 


some moderate great 


importance importance importance 


Importance 
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Response Key 


1 


very limited 
importance 


Criteria 


Shares workload on 
faculty committees. 


Communicates effec- 
tively with peers. 


Progress of his/her 
students in providing 
individualized nursing 
care. 


Progress of his/her 
students in becoming 
self directed in 
learning. 


Sets an example for 
his/her students. 


Follows institutional 
policies. 


Proposes needed change 
within the work environ- 
ment. 


Follows through on 
commitments. 


Progress of his/her 
students in recognizing 
their strengths and 
limitations. 


Progress of his/her 
students in functioning 
as members of the 
nursing team. 


Progress of his/her 
students in recognizing 
their responsibilities 
as future members of the 
nursing profession. 
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III. Comments 


Any comments which you have concerning nursing instructor 
evaluation and/or this particular questionnaire would be 


appreciated. 


Thank you for completing this questionnaire - your input is 
appreciated. 


APPENDIX B 
CORRESPONDENCE 


158 


159 


10926 - 129 Street, 
Edmonton, Alberta. 
T5M 0X9 


December 22nd, 1976 


Ms. Shirley Shantz, Chairperson, 
Consortium of Nurse Educators, 
Red Deer College, 

Red Deer, 

Alberta. 


Dear Shirley: 


I am presently enrolled in the Masters Program in Educational 
Administration at the University of Alberta. I am looking at 
nursing instructor evaluation as the subject of my thesis. I would 
appreciate an opportunity to attend the next meeting of the 
Consortium of Nurse Educators which I understand is scheduled for 
Friday, January 28th, 1977. At that time I would like to explain 
my thesis plans in detail and request the cooperation of the 
Consortium members. I believe this would take about one half 

hour and I would appreciate being able to speak with you in 

the morning if possible. 


Thank you for your assistance. 
Yours sincerely, 
Lee Cadman 
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FACULTY OF EDUCATION 
DEPARTMENT OF EDUCATIONAL 
ADMINISTRATION 


THE UNIVERSITY OF ALBERTA 
EDMONTON, CANADA 
T6G 2G5 


10926 - 129 Street 
Edmonton, Alberta 
February 9, 1977 


Dear 


I was pleased to have the opportunity to meet with you at 
the January 28th meeting of the Consortium of Senior Nurse 
Educators. The input which I received from your group was 
of value to me. 


I will be circulating my questionnaire on Nursing Instructor 
Evaluation within the next three weeks. I would appreciate 
your assistance in the following ways: 


- I would ask that you send me a list of your 
faculty members whom you consider to be teaching 
one half or more of a full time position in your 
Diploma/Baccalaureate Program. Please include 
those employed both sessionally and on a permanent 
basis. 


~ I would then like to send an introductory letter, 
a questionnaire and a follow up postcardto the 
above faculty members utilizing their work address. 


Many thanks for your cooperation. 


Sincerely yours, 


Lee Cadman 
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Request For a Study Summary 


Dear Ms. Cadman: 


Please send a summary of the results of your study on Evaluation of 
Nursing Instructors to: 


Name 

Street 

City 

Postal Code 


Reminder Postcard 


DEPARTMENT OF EDUCATIONAL ADMINISTRATION 
UNIVERSITY OF ALBERTA 


Dear Nurse Educator: WAVE, 


Recently a questionnaire on Evaluation of Nursing Instructors was 
sent to you. If you have already completed the questionnaire, I 
would like to take this opportunity to thank you once again for 
your assistance. If you have not found the time as yet to respond, 
I would request that you do so at your earliest convenience. Your 
input is appreciated. 


Thank you for your cooperation. 


Sincerely yours, 


Lee Cadman, R.N. B.Sc. 


161 


he CE SE — 
‘ 


ao mtn nities eae) peal ole fh 
. 


ee ee eg Se ES A A TT ce: IIR i 


F bi Yo worded oo orbnome desu 


‘oa? lide famud wah wa the 
par J Ae Pini mest OY mae” 

® SeGht-tae study 3 
Pru, yl orennt? 


32,8 WH onanitiga ae] 


162 


FACULTY OF EDUCATION 
DEPARTMENT OF EDUCATIONAL 
ADMINISTRATION 


THE UNIVERSITY OF ALBERTA 
EDMONTON, CANADA 
T6G 2G5 


Apr ale uel 9 7 


Dear Nurse Educator: 


Several weeks ago you were sent a questionnaire concerning 
the Evaluation of Nursing Instructors. To date, I have 
received responses from approximately 65% of the Alberta 
nursing instructors selected to participate in my study. 

I believe that the quality of the research would be 
improved if the perceptions of more of you were represented. 
If you are willing to participate in the project and have 
not yet done so, I would appreciate your completing and 
returning the questionnaire within the next few days. 

(I have enclosed a duplicate copy of the questionnaire 

in case you have misplaced the original). 


If you have already sent in your response, please accept 
my thanks once again. 


Sincerely yours, 


low Caclman) 


Lee Cadman, R.N., B.Sc. 
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